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; . '* ^ FOREWORD 

'^his report presents the results of a 2-year project to design, pretest, and revise the Statistical Survey of Elementary 
Sclj^ols, which is intended to provide information on federally funded elementary education programs operating in 
loC^l public school systems. The report also documents the activities and decisions which brought about these results. 
//This project was undertaken jointly by RMC Corporation ami fntran Corporation of Minneapolis, under Office of 
;|^ucation Coatract' No. OEC-0-72-5224. Key Natiopal Cenjer^for Education Statistics (NCES) personnel responsible 
''/for the conduct of the study were John Sietsem^, Lois Green, and.,Susan Hill. 

Other DHEW staff providing guidance were Harold Nisselson (NCES), Nancy Bradley Royall (BESE)^ and Lois 
Elliott (BEH). Jimmie, Fortune of Virginia Polytechnic Institute and David Berliner of Far West Regional Edticational 
Laboratory acted as consultants. ' . 

Special thanks go to those in State apd local education agencies whos^ cooperation was essential in carrying out the 
project. Of great assistance was the Committee on Evaluation and/ Information Sy^stems of the Council of Chief State 
•School Officers, and particularly Jerry Barton of Texas and Qyde Bezanson of Minnesota. Special mention is merited 
by the 594 respondents-teachers, principals^ school district coordinators-who not only completed questionnaires but 
also contributed valuable and constructive comments. 
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Chapter ! 

OVERVIEW * . 

"• * ♦ . I 

This report is the result of RMC Research Corporation's contract with^the National Center for Education Statistics . 
(NCES) to redesign the Elementary School Survey (ESS). The last ESS was conducted in the spring of 1971. During" the 
course of this contract with NCES, the name of the survey was changed to the StatisticaP^urvey of Elementary Schools 
(S§ES)and it will be referred to in this manner throughout this report. ^ 

With the passage of the Elementary^nd Secondary Education Act of 1965 (ESEA), Congi'fiSs initiated a serie*^ of 
legislative actions designed 'Ho meet the special educationalneeds" of certain specific groups of children. Among tljose 
who have' received the benefits of this legislation since that time are cducationally^ep rjved children fro m low-income 
families, children of migrant agricultural workers, .neglected andcdelinquent children, children from^homes where the 
primary or dominant language is not English, and handicapped children. 

The ESS began in. 1968 as a compensatory education survey and has grown in importance over the years. Tlie new ^ 
SSES is specifically designed to help piogram planners witjiin the Bureau of School Systems and the Bureau of^ 
Education for the Handicapped (BEH) j^nswer the basic question ^f how well the education community has compHed 
with the intent of ESEA legislation iit making various special programs available to the children for whom they were 
Jr designed. However, the survey is not an evaluation as^uch but rather provides baseline descriptive data on programs 
provided by ESEA titles 1, Hi, and VII, as well as programs for the handicapped. In addition information is obtained on 
several other areas, including pupil mobility, teacher characteristics as related to pupil performance, and edocatioiial " 
broadcasting. The first two of these areas are new to thisVsurvey and appear at the request of the Nationjjj^ Institute of 
. Education (NIE). Thfe last is of special importance to NCES. 

The SSES was initially part of a larger data-collection system that evolved from an agreement between the U.S. 
Commissioner of Education and the Council of Chief State School Officers. As a result of this agreement, the SSES 
represent^a major effort to consolidate the data requirements of all the bureaus and offices within the U.S. Office of 
Education (OE). With this in mind, an attempt was made early in the redesign stage to eliminate potential data overlaps 
between this survey and other data collection efforts. 

As originally planned^ the survey was .designed to produce a set of data elements linked to policy questions that had 
been developed by the New England School Development Council (NESDEi^). This linking process was to result in new 
questionnaiTCs~{hat Avould be pretested in 50 school districts drawn from a number of Stales that had volunteered to 
participate. At the beginning of the study, sbc major tasks were envisioned: / 

• Task 1 : Policy Question Review 

• Task 2: Instrumentation 

• Task 3: Field Test 

• Task 4: Analysis Plan 

• Task 5: Logistics Plan 

• Task 6: ReliabUity and Validity Study ^ 

The original contract for this study began in July 1972. Intran Corporation, working under subcontract to RMC, 
began work a't the same time. The study was sche13uldd to end in June 1973, but NCES postp^nenent of the spring ^ 
1973 pretest resulted in an extension of funding and time until September 30, 1974. 

Work progressed under •the organization of tasks described above through the early spring of 1973. Task 1 was 
completed and a report on this activity was submitted in November of 1972. Following the process of establishing data 
elements, six sets of questionnaires were developed from Novemberi»'through February of 1973. Tliese consisted of the 
first draft questionnaires, followed by five complete revisions. Each set of questionnaires involved four instruments^^ 
^ . .. . — 
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district, school, teacher, and pupil qvcstionnaircs. After what was thought to be the final revision, an 0MB clearance 
package was jrUbmitted to OE in February of 1973 and plans were begun to pretest the survey in the spring. * 

However, because the questionnaire development and revision process had taken considerably longer than expected, 
iiUKufficient time remained to prepare for a proper pretest, fn addition, NCES had surfaced somc\questions concerning 
the intent oryhfe technical sujp^porting statement in the^>Office of Management and Budget (bM6) clearance package 
that, if left urtanswe/ed, woiml delay the clearance process. Consequently, f^CES made the decision to postpone the 
pretest untii at feast the fall/Of 1973. The reasons for the postpdnement were generally as follows: 

• prolonged ihstrumentation and revision of questionnaires, o " 

• NCES qucsinpn^ concerning portions of the 0MB Submission, 

• insufficient tihie to prepare a revised 0MB package and receive OE and 0MB clearadce early enough to pript and 
mail survey packets to SSES survey coordinators prior to April 6 (the last date schools could receive the 
questionnaires and still be expected to complete them before the end of the school year), and 



; end o\ 

• internal discussion within OE as to the relevancy of the SSES in lijit of uncertain program funding. 

Tlie contrattor was notified of the postponement on March 23, 1973, and at the same time was asked to prepare a 

revisfii^^lan for pretesting the instrumfitttsandproccdures-in thefall^ 1973. ~ - 

'^^^Tl^ of the Elementary School Survey was submitted to NCES on April 13, )973. This 

plan actually encompassed suggestions for two basic approaches to the pretest. Plan 1 involved little more than carrying 
on with the strategy developed to date, revising some of the methods for testing alternatives in the field during the 
pretest, and establishing the fall of 1973 as the target date for pretest operations. 

Plan II was much broader in scope, During the course of the contract, it had become apparent to all involved that* 
the NESDEC policy questions used in the design of the questionnaires were outmoded. A more flexible approach to the 
policy question concept was nceded-^one that would meet current data needs and that would be easily adaptable in the 
future for changing legislation. Tliis last point was particularly important in light of changes in ESEA legislation that 
wcrortmminent at the time (i.e., the proposed Better Schools Act, Education Special Revenue Sharing). Plan II 
proposed to identify new policy issues and chzmge the thrust of the data collection to a pupil target group orientation 
instead of the program-related emphasis that had been used until then. 

During the ensuing months a totally new plan that combined plans I and II was developed. NCES felt this new plan 
was the one that would receive the most favorable acceptance from 0MB. Concurrently, details of the technical 
approach for the development and analysis of the pretest were also being worked out with NCES. This development and 
evolution continued until August of 1973, during which time the^ontr^ctor modified and updated questionnaires and 
prepared a general statement of contract plans for the coming year. Since considerable revision of the questionnaires was 
anticipated during the next few months, the pretest was once again postponed-this time until the spring or i974. 

An intensive review of the study and the revised questionnaires occurred during meetings held August 29 
and 30^ 1973, with the Subcommittee on School Surveys of the Committee on Evaluation and Information Systems 
(CEIS). Following tlie CEIS meetings, RMC begtxn another round of questionnaire reviews and revisions that extended 
well into November of \973. Numerous discussions and meetings took place with data users such as BESE, BEH, NIB, 
and NCES. At the same time, the comments and suggestions of the CEIS members were considered during the revision 
process. Wliile the new questionnaires maintained the original program-related approach, new questions were developed 
that reflected current policy issues and user needs, as well as a greater emphasis on identifying and counting relevant 
pupil tatget groups. In addition, questionnaire, logistics, and sampling alternatives were being developed*for thd pretest. 
These alternatives are defined and analyzed in later chapters of this volume. 

In Oqtobcr 19^ NtES and the contractor jointly agreed on a more precise definition of tasks and the direction the 
study vyould take. Tliis was reflected in RMC's Revised Technical Proposal of October 5, 1973. In general, the tasks fell 
" into three main categories with a grouping of subtasks for each? The major categories were: instrumentation, field test, 
and developing recommendations. Figure 1 shows the October 5, 1973, project schedule and outlines the subtasks. 

When the contract was amended and the time extended to complete the new tasks, RMC outlined goals for the 
pretest, which were to: V ^ . 

(1 ) supply information on how well -alternatives, instalments, and procedures worked when implemented; and 

(2) moke recommendations on how the instruments and prpcedures could be revised for the full-scale 
implementation. ' , ' 
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) dchicvc these goals, six major objectives were established for the prefest. These objectives were to: 

(1) collect information that would be used to fefine the instruments and ^rocedures'so that they would be cle^rarid 
Unambiguous, 

(2) develop and test alternatives for obtaining a random sample of pupils using procedures<qapable of being audited, 

(3) develop auditable procedures for obtaining a stratified sample of schools and for oversampling pupils, 

(4) 4evelop alternatives for selectinglhe most suitable respondent for certain data requests, 

(5) collect validity data on selected items, and , . » 
(6\jsefine the instrunjents and procedures so they would provide meaningful and interpretable.data that would meet 

^the needs of clients served by NCES. 



These objectives are described in a d,ocument entitled Pldn^r the Pretest of the SSES, dated December 12, 1973. 

The pretest plan outlined how the six objectives were to%,e accomplished and served as a guidebook for conducting 
the pretest. Tables were presented for each objective describing: 

(A) the objective itself, ; 

(B) the method to be used in achieving the objective, 

(C) the analysis needed, and 
recommendations. 




the 
as an 



Through tTie use of a coding system, each table was linked to a technical appendix that gave full details for of 
four points above. T^e tables wore designed to provide an overview of the pretest in a concise format and to serve ai 
index to a detailed technical discussion of each activity. 

During t)ye time that the pretest plan was being developed, questionnaire revisions were completed and included as 
part of aiKOMB package that was submitted to NCES on November 30, 1973. WhUe awaiting clearance (which was 
received on March 20, 1974), ftirther preparations were made to put the survey in the field. Mailout of survey materials 
took place on April 12, 19H Validation and sample check site visits to 20 participating LEA's began in early May. 

The remaining chapters in this volume detail the activities described in general here and present the major results of 
the pretest. 



10 



ERIC 




Chapter 2 . 
QUESTIONNAIRE DESIGN 



INTRdDUCTlON 




Qyicstionnairc development for the SSJS-caTTEenboked at in twojlminct stages. The first stag(ijvas basically a 
redesign, of the 1971 ESS. Qu/isUonfiaire. design was structured aj^d^licy questions tiJaT^RTAieveloped by 
NESDEC and approved by '^the'^U.S. Office of EducaUon (OE). An attempt was also>rnade at thispointlo rtunimize the 
replication of data elements requested on other surveys, such as the Consolidated Program Information Report (CPIR) 
and Elementary Secondary General Information Survey (ELSEGIS). In-this way, it was hoped to reduce the response 
burden on LEA^s 4>articipaU^g in more than one Federal survey and thus encourage cooperation for the SSES. As a 
result of these efforts, eaf^ draft questionnaires revealed a strong link between data elements and the New England 
School Development Council (NESDEC) policy questions. 

However, during the second stage of questionnaire development, a significant change in emphasis occurred. Just 
prior to the postponencient o( thS pretest (from spring 1973 to spring 1974), the questionnaires had begun to show a 
definite user-need orientation with less emphasis on the NESDEC policy questions. Since the postponement allowed 
more time for questionnaire revision, it was decided to develop the user-need approach in more depth. Questions were 
also added or redesigned to obtain data on pupil target groups. This last shift came about as a result of then impending 
legislation: the proposed Better Schools Act, Education Special Revenue Sharing. 

The following sections of this chapter describe the events leading up to the final pretest questionnaires that were 
implemented in April 1974. 

QUESTIONNAIRE DEVELOPMENT PRIOR TO PRETEST POSTPONEMENT 



Policy Question Review 

SX the onset of the study, the NESDEC policy questions were the central force behind the redesign of the SSES. As 
originally envisioned, the NESDEC policy questions were to reflect data requirements at all levels of decision-making on 
eight federaUy funded programs: ESEA titlcS I, II, HI, VII, and VIII; NDEA III; VEA 68; and EHA, part B. 

The NESDEC policy questions for each program were divided into four general areas: 

A. To what extent are program funds appropriately targeted? 

B. Are services addressed to the special education needs of participants? 
'C. What effects are associated with the program services? 

D. Is th^re a need for change in the Federal and State conduct of the program? 



Within each of these broad areas wcte several major policy questions. These, in turn, included linkage questions 
describing the data required to answer the major policy questions. 

One of the first tasks in the study was to determine whether NESDEC's policy questions were applicable for an 
elementary school survey and whether they reflected the current information needs of the bureaus and programs that 
would use the data obtained from the survey. To accomplish this task, Mr. Absalom Simms and Dr. Charles Hammer of 
NCES/DIS arranged interviews for RMC with the following people: 

• Miss Nanty Bradley, BESE 

• Dr. Lois Elliott, BEH ^ 
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• Dr. Milbrey Jones and Mrs. Gladys Pcndcrgraph, OLLR 

• Mr. Lcroy Cornclson and Mrs. Thcda Cottd^, BATVE 

• Dr. Hyram Smith and Dr. Jerry Reynolds. NCIES 

• Mrs. Barbara Resse, NCES/DSIS 




llTcof the si/rvey-its strengths and 
cation that -some of the eiglU pcograms 
It was also pointed out that several of 



Hiese intdrviews solicited comments on the NESDEC policy questions, 
weaknesses, and additional data needs. The interviews also provided the 

under consideration -werd not relevant for an elementary schooLp , _ _ , 

NESDEC*s policy questions could not be adequately answered^^part of a mjulout survey. In addition, some doubt 
raised about the inclusion of ESEA^title 11 (school Whxj/y resources) and,' NDEA III (strengthening instruction in 
science, mathematics, and foreigh language|Ht(2cause fyiWing for these prog|:ams in Fiscal Year 1974 was uncertain at 
that time. 

Early in the study, NCES and the contractor TRfiJC) also established contact with the State education agencies and 
initiatedjState interaction on this survey that continued throughout the next/two years. On August 7 and 8, 1972,'RMC 
staffWmbers attended the Joint Federal State ^ask Force on the ESS in Louisville, Kenutcky (at the request of 
NCES). This meeting provided the contractor with background information on the evolution of the survey and 
stimulated useful discussion about State priorities^ndjdata needs. / 

After meeting with the State representatives, the next step in questionnaire development was to eliminate those 
portions of the NESDEC policy questions that were not applicable to an elementary school population. When NESDEC 
had developed its cornpendium of policy questions, the orientation was toward a secondary school population. 
Therefore, certain programs and data elements were not appropriate for an elementary school survey. After reviewing 
the policy questions and furthet discussion with OE representatives, recommendations were made by the contractor to 
exclude the Dropout Prevention Program (ESEA title VIII) policy questions as well as those concerning fhe Vocational 
Education Amendments (VEA 68). Other data elements were also dropped from the remaining program policy 
questions because of their secondary school orientation. . 

Finally, several sections of the NESDEC policy questions-namely those dealing with 'the impact of Federal programs 
and program management-seemed beyond the scope of a mailout survey such as the SSES. To satisfactorily address 
these areas, a research design was needed for specific program impact measurements.. This type of research is generally 
the thrust of special .studies associated with the Office of Planning, Budgeting, and Evaluation (OPBE) of OE. 
Therefore, it was recommended that these areas be excluded from the SSES, and NCES'qSncurred. 

Once the new "streamlined" policy questions were agreed upon, the contractor began the job of linking the 
remaining policy questions to data elements and determining the analysis jequirecj tb answer them. This effort was 
completed for ESEA title, L II, III, and VII; EHA, part B;and NDEA III. The contractor had previously recommended 
tlie elimination of ESEATl and NDEA HI from the new SSES because there'was considerable doubt about whether 
funding for these programs would continue. They were subsequently dropped. 

When the linking task was finished, data elements were grouped by linkage questions and a second wave of 
interviews with BESE and BEH personnel was conducted. Since all of the data elements identified by the policy 
question review could not be included in the survey instruments, the bureau representatives were asked to identify 
those elements that had low priority or were not useful at all. In this way, the contractor was able to document the 
"keep" or "drop" status of every data element and thus completed the first step in preparing draft questionnaires for 
bureau and NCES review. ' 



Questionnaire 'Development 



In October of 1972, the contractor began developing items and drafting questionnaires. In keeping with the former 
Elementary School Surveys, the new questionnaires addressed f6ur levels of response-district, school, teacher, and 
pupil-with separate instruments designed for each. (The pupil questionnaire, although requesting information about 
individual pupils, was to be completed by the teacher.)*Within each questionnaire, the first section asked general 
questions about the district, school, teacher, or pupil and then moved on to separate sections regarding ESEA titles I, 
III, and VII. At this, time, the last section on each instrument collected information on Education for the Handicapped, 
part B. 
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EarB^r in the contract, the contractor bad been informed that questions relating to educational/technology, the 
environment, and educational manpower would h§ included as part ofthe survey. However, since these areas were 
outside of the policy question framfework, there was some question aHi^lime.as to whether they would appear on the 
survey. Eventually, <item^ were drafted for inclu^on as/the final sections on the questionnaires. But the future use of 
thesq^questions.wasr^rtot clear until much later in the^t]tt^y. 

By late December 1972, the third draft of the questionnaires ^as submitted to NCES. However, meetings were still 
being held with BESE and B EH representatives and revisions were still occurring. ' 

The Shift from Policy Questions to User Need * 

The primary purpose of the instrumentation task was to produce questions that could be linked to the NESDEC 
policy questions through analysis. This was not always an easy task. Writing items for the type of data outlined in the 
policy questions was difficult because the intent of the' linkage questions was sometimes vague. In addition, the 
NESDEC policy questions had been developed'by means of formula questions applied tg all programs. As a result, the 
policy questions were bften repetitious across programs. To avoid making the questionnaires equally repetitious, data 
elements required by all of the programs were iden,tified and placed into, the general section of each questionnaire 
wherever po^ble. A good deal of time and attention was also given to devising routing questions so respondents would 
not have to/work through items on Federal program^s that did not apply to them. Other individual data elements were 
revised or /collapsed into one- question in a further attempt to reduce the repetition fot^nd in the policy questions. 
Finally, each revision of the instruments meant complicated changes to the docdmerf%tion the contractor was ^ 
preparing that showed how each questionnaire item related to a general area, policy question, linkage question, or client 
request. 

In mid-January 1973, the contractor met with bureau representatives to discuss thie latest draft of the 
questionnaires. As discussed above, considerable time had been spent in producing documents that distilled the essence 
of the NESDEC poliay questions into data items for the four questionnnaires. An item-by-item review vvith the bureau 
representatives was planned. Some additions, revisions, arid deletions were anticipate<l, but the contractor did not expect 
i\ik thrust of the redesign approach to change. However, the outcome of this latest meeting was a fifth draft of the 
ihsfruments that reflected a shift in the methodology for designing new SSES questionnaires. 

^ 'buring the January meetings, the bureau representatives indicated that they wanted an increased emphasis placed on 
data needs that were of current interest to them. These new needs were very different frohi those found in the 
' NESDEC policy questions, this was clearly demonstrated by the dropping qf questionnaire items that did not solicit the 
" type of information currently w^ted (even though they were major policy question links) and the addition of new 
items that reflected current data nee^s. What the bureau representatives seemed to be saying W£^^ that some of the 
" policy, questions were no longer relevarit>and that there wejf-e more pressing data needs that should be satisfied on a 
mailout survey to schools and school districts. , . 

For example, BESE indicated that^it would like some items that appeared^on the earlier forms of the ESS (1970 and 
1971) asked again to see how certain Federal programs had changed over time. Additionally, BESE was in the process 
of analyzing the 1970 ESS data and added questions to pursue more fully some of th^arly findings there, hi some 
cases this meant simply using questions from the earUer ^SS; but in others questions were totally rewritten with the 
intention of collecting comparable data. This approach to questionnaire de^Jgn was quite different from the. one the 
contractor had been using and resulted in extensive^vision of all foiir questionnaires. ' 

Toward the end of this instrumentation stage, BEH also shifted its frame of reference. Prior to this, the contractor 
had developed the sections of the questionnaires dealing with handicapped children from the viewpoint of collecting 
/ information on programs and services funded under part B of the Educgition for the^ Handicapped Act. But at this time, 

, BEH indicated that wha*t they, really needed was more information about any programs and services for the 
handicapped. As a result, tlie instrumentation for thes6 sections began td grow n\g[t# general in nature, collecting data 
on State -arid local .ptpgrams for the handicapped rather than limiting questions to s'ervices funSded by EHA, part B. But 
in> anothei;, sense, the handicajjped sections of the questionnaires'^had become more specific. Where early drafts asked 
about handicapped children in general (as implied by the policy questions), BEH now requested that questions be 
answered by type of handicap wherever possible. For example, on sugh a general question as "How many handicapped 
^ pupils attend this school?" respondents were asked to indicate numbers of pupils in specified grades by 12 
handicapping- conditions. This created considerable revision of the questionnaires and added commensurately to the 
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time spent on. instrument design. And" finally , BEH requested the addition of questions concerning preschool programs 
and services for the handicapped. This was a high-priority area for BEH during the 1972-73 school year, but the policy 
questiorts had not included preschool services at all. * •< 

The contractor wa^stUi making major changes an^addition(s)to the questionaires that reflected current da|a needs 
and changes in priorities as late as February 16, 1 973. Deletions, revisions^ and additions were made, by bureau 
representatives without respect to analysis issues or the NESDEC policy questions. At this point,'the contractor realized 
tha(;Ae was attempting to accomplisli two completely difjerent goals. On the one hind, he was trying to be responsive 
to NCES by redesigning the survey within the conceptual framework of the NESDEC policy questions. On the other 
hand;^BESE and BEH had some requests for data that were extremely important to them but that did not fit into the 
NESDEC policy question' framework. Consequently, the contractor w^s placed in the position of developing 
questionnaires that no longer met the rigid outlines of the policy questions but instead reflected current Federal 
program issues. . 

Moreover, an addifional but related problem was identified concerning the analysis plan for the surv)s^. The 
contractor could develop an analysis plan for questionnaire itemrbased on the NESDEC policy questions, but it Would 
be difficult to develop one for the new items becau'se the rationale for their inclusion was not always known-beyond 
the fact that they satisfied current data needs. \ ' ^> ^ • ' V^^ 

■ . ■ :i . \ ^- ' 

QUESTIONNAIRE DEVELOPMENT AFTER PRETEST POSTPONEMENT 

By February 1973, four new draft questionnaires-^totaling 234 items had been developed. These questionnaires 
" received t^e verbal approval of BESE and BEH representatives and an 0MB package was prepared and submitted for 
NCES approval. However, for reasons discussed in chapter I , the spring 1973 pretest was postponed and the contractor 
was asked to begin work on revised plans for conducting the stCuly. 

An early. ^tep in all of the proposed pretest plans was further revision ai<^ reTinement of the questionnaires. In 
preparation for these revisions, the draft questionnaires wereXdistributed^ within OE and to State representatives 
(including^CEIS. members) for review. In addition, the contractor, was asked to attend a two-day meeting (in August 
1973) of Surveys Subcommittee of the Committee on Evaluation and InfQrmati9n Systems (CEIS) where all four 
^ questionnaires were reviewed with the committee members in 'fr^at detail. CEIS contributed many helpful suggestion^ 
that were carefully considered in subsequent questionnaire workshops. During theXEIS meetings, the contractor was 
also introduced to the data requirements of the National Institute of Education (NIE). A representative from NIE 
discussed existing questionnaire items that would ,b6 of interest to NIE as well as the possible addition of other 
questions. * 

Since the postponement of'the pretest allowed tlie time necessary for more adequate response to user needs as well 
as issues raised by pending legislation, plans were begun to conduct questionnaire workshops with NCES and bureau 
representatives. However, because of scheduling -constraints, only BESE representatives and NCES technical staff 
members attended the first 3-da'y .workshop in September 1973. Over the 3-day period, BESE set item priorities and 
provided an item-by-item justification fpr the inclusion of its questions in this survey. The BESE sections of all four 
questionnaires were analyzed with an eye to producing revisions t|i^ would satisfy the bureau's current data needs. 
' Questions wer& also added or rewritten addressing legislative issues such as the Quie proposal and the proposed Better 
• Schools Act, The comments resulting from the CEIS meetings arid the earlier distribution of cjuestionnaires to other 
State and OE representatives were also weighed as part of the discussion. 

After the BESE questionnaire workshop, meetings were scheduled and held with NIE staff members. These resulted 
in the addition of several new questions on the teacher and pupil questionnaires. Item priorities and justification'for 
their inclusion were obtained from NIE. ' 

HoNvever, scheduling meetings with BEH proved to be difficult. The contractor's original contact there. Dr. Lois 
Elliott, had left the bureau and /nother BEH contact for the SSES had^ not yet been appointed. When a BEH 
representative was identified, rp^etings were planned to finalize the portions of^the questiormaires dealing with 
' programs for the handicapped /The initial meeting with BEH provided the basis for minor revisions but an item-by-item 
justification could not be obtained since the new representative was not familiear with the history and development of 
the BEH questions. Although considerable effort and time were expejidfeiS by RMC staff members of the contractor to 
elicit additional BEH iftput to the questionnaires, attempts to schedule other meetings were unsuccessful. Subsequent 
contributions were niptde primarily thorugh telephone conversations (initiated by RMC) with BEH staff on specific » 
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estions. As a result, important issues raise^d by^C. CEIS. BESE. and other persons within NCES. and the OE 
regarding the length and complexity of the handic^ped sections were never completely resolved. 

During the period. NCES also requested that certain other items appear on the' revised questionnaires. These 
included questions that would collect supporting information for a previously conducted staffing survey of school 
personnel who "provide instructional or related services to handicapped children. Jn addition, the debate about the 
questions dealing with educational technology and environment was raifsd again. RMC was opposed to the inclusion df 
•these questions because they were inappropriate for a survey such as the SSES. Further, reasonable justification had 
never bQ^n established about the usefulness of the data that would be obtained from them. The discussions between 
RMC and NCES on tliis issue, continued for several weeks'; extending the questionnaire revision timQ. In the end, 
however, the environment items wer^. dropped, and only the educatic^nal technology (broadcasting) Questions were 

included. ^ 

When it was planned, the final revision process f(5f the questionnaires was to./iave been quite short. But as can be 
seen from the description above, in reality, it was a time-consuming procedure that resulted in virtually rewriting all 
four questionnaires. However, the basic program-related organization of the questionnaires remained the same as 
' described earlier, with each instrument having a general section and separate sections for ESEA titles I. III. and VII; 

programs for the handicapped; and educational broadcasting. Hie exception to this was the pupil questionnaire where 
V the final section collected information on pupil achievement test scores as requested by BESEs NIE. and NCES. 

On October 19. 1973. annotated drafts pointing out data uSers and u;iresalytfO«?blem areas wer^ given to NCES 
for final technical review and written sign-offs prior to 0MB sup^m^ion. However, ctoments and-sign-offs from the 
reviewers were slow in coming. Consequently. RMC was still re^i^ing que^ns^andj[^^ on the questionnaires as 
late as November 23. 1973. While these last-minute changtKwere being i!ift^<1mb>was also preparing an OMlB 
clearance package. The final questionnaires Were indued inYtae 0MB packageptd-s^ to OE on November 30. 
'M973. The num.ber of questionaire items in the four qVs^nn^r^^ad^ncr^ frfem 234 (the number oh the last 
drafts prior to pretest postponement) to 309. ^ ^ ^ * 

SUMMARY * - . 

In ^retrospect, now that this study is over. tk. task of designing questionnaires based on the NESDEC policy 
questions was not'as useful as originally envisioned. Se\eraLfactors contribute to this point of view: 

• (1) The NESDEC policy questions were developed with^a secondary school population in mind. Even though some 
Federal programs serve aU 12 grades (e.g.. ESEA title I), the major thrust of the policy questions was toward the 
upper grades. As a result, it was difficult^g. relate Some program areas to ah elementary school population. 

(2) The NESDEC policy questions followed a^andard formula in their development that did^not allow for af\e 
inclusion of current data needs. This also caused some data elements appropriate for one program to be included 
in other programs where they had a tenuous relationship at best. 

(3) The NESDEC policy questions were finalized in the fall of 1971. When questionnaire development began in late 
1972. some of the areas found in the policy questions had cahnge^ in importance because of shifts in Federal ' 
funding or the need for data. Additionally, current data nee^ became apparent as the study progressed that had^ 

' not been evident when NESDEC completed its work. 

These three factors go a long way toward expl^ning why only 31 percent of the items o^i the questionnaires pretested 
this spring had a link, to the NESDEC poUcy questions. , : 

It would appear that the particular policy question approach used in developing the SSES questionnaires was 
time-consuming and expensive given the benefits derived. But this is not to say that questionnaires can or should be 
developed without respect to key issues or major questions that need to be answered. Rather, a broad framework of , 
policy questions should be developed that is flexible enough to accommodate a variety of Federal programs as well as 
the results of changing legislatiotf and user needs. . ^ 
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Chapter 3 / 
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MAILCei SURVEY / 



INTRODUCTION / 

/ After submitting the 0MB clearance package to U.S. OfHce of Education (OE). the next major step i" the Prete^* 
toward which RMC directed its efforts was the preparation for the mailout f ^ey. RMCs Pton /or ft-^re^^^ 
Si was developed and submitted to NCES on December 12. 1973. This plan outlined the strategies that would be 
eSloyed Tdev^oping and analyzing the alternatives to be tested in the, field. During the next few moMhs the 
contraSor prepared the survey support materials, selected the States and LEA's that would participate irt hf pretest, 
developed fheTernSwL tha't would be tested'and analyzed (see chapter 5 and 6). determined the mailout survey 
logistics (described in chapter 4). and conducted the precanvass of participating districts. ^ , 

> ■ ' 'i 

SELECTION OF PARTICjeANTS 

States • 

A total of nine States agreed to participate in the pretest that would take place wi the spring of 1974. Two of these 
■ States had volunteered at the CEIS meeting the preceding August; others who had agreed to participate in the pretest 
planned for 1973 were asked to participate again. When these two procedures did not produce enough participants 
NCES c^L od^er States to request their participation. Tlie Stp^s that were asked were selected on the bas^ of 
preSous gooperation with OE and characteristics such as urbanism. presence of significant numl^ers of certoin minority 
popXtions. and expected presence of ESEA titles .I'dncluding Migrant). III. VII. or any programsfor ^he handicapped. 

" Districts 

When the States were contacted, eich was'asked to volmh^er 5 o 10 LEA^ for participation. Some States agfe^d to 
preS tl« SSES in the schools in the participating districtsirefer\ed to as "fulT' participation); whereas oUier S a te 
a«eed ody to pretest the district questionnaire in the participating districts. This was done to supplemen the otal 
nSer o district questionnaires in the pretest (referred to as "district^nly" participation). In addition, otjier States 

. had a mlxLre of both types of districts. 'After a State agreed to participate, the State survey coordinator was given a bst 
of LEA'S that RMC had selected as preferred districts for the pretest. These lists were compiled by electing the 
?972 CPIR sample (the only available source of program participation information) the districts in each S aff that 
nduded as many of the targef programs as possible and represented variations in other f '^^^^^'^harac^'^*^^^^^^ 

- contractor conducted telephone interviews with the designated representative in each of the nine States THese calls 
were uS to brief the sfates on the details of participation, to finalize the district sample in ea^h fte jnd « 
Srmine. whether each State would allow direct shipment of pretest materials to the LEAs Florida was the only State 
that agreed to the latter procedure. The list of participating States and LEA s is shown in table 1 . 

Summary ^ 

«r Number of LEA's 

Full District only 

Arizona i ' , ^ " . 

Colorado • • ^ 

Florida ^ ' ' 

Kentucky • ^ • I 

Michigan ' " . 

Minnesota * ' 

New Jersey / , - " * 

Texas . . . ^ 
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Table 1 .-Participating States and LEA's 



State 


LEA 


Number of 
schools i> 


Comments 

A- 


Arizona 


Mesa 


J 






Nogales 


■a 
J 


Supplementary sampling 




Somftrtnn 


1 

1 ^ 


Supplementary sampling 




Wilson 007 ; 


2 


Supplementary sampling 




^ Adams County 


3 






Boulder Valley 


3 


Supplementary sampling 




^uiuiiiuo dpriiigs 


J 


Supplementary sampling 




Ft. Lupton 


3 


Supplementary sampling 




Greeley 


3 


Florida 


Rrf*i/n rH tr\\ \ nf a/ 

oicvuiu ^ouniy 








piUWalVl WOUiliy 










J 






Dade County 


3 


Supplementary sampling 






J 


Supplementary sampling 


Kentucky 


Campbell County 


' 0 


District only 




Carlisle County 


0 


District only 




Canoll County 


0 


District only 




Clay County 


3 




Fiyette County 


3 






Fulton County ' 


3 


Supplementary sampling 




niu^niuji L«ouniy ' 


3 


Supplemental^ sampling 




ruuucdji L«iiy 


3 




runiKviiJo L.iiy 


0 


District only 




auiTicrsci LAjuniy , 


3 




Trimble County 


0 


Liuinci oniy 


Michrlmn 


l<ntrtn Dantrl* 

i:>uiUii ivxpius 


J 






vjimiu txupius 


J 


• 




HnlfnnH P{»u> 

nouunu L.i(y 


3 








3 






Vr\ n f 1 n 

rOJl llaC 


3 




Minnesota 


Albert Lea <| 


0 


District only 




Blootnington 1 


0 


District only 




Blue Earth 


0 


District only 




Breckenridgo 


0 


District only 




Crookston 


0 


District only 




Hopkins 


0 


District only . 




MlnnCiinnHv Qnpnlnl 


u 


District only 


Moorhcad ^ 


0 


District oniy 




iNortn 2>t. raui 


0 


District only 




Owatonna 


0 


District only 




RosoWUe 


0 


Liisiiiui oniy 




Auantic L.ity 


3 


Supplementary sampling 




LAJWilC IUWil3iUp 


*> 




Pater son 


3 






Union C\\\t 








V mcuuia L.iiy 


3 




Texai 


Alice 


3 






Austin 


0 


District only 




Brownsville 


3 




Dallas 


0 


District only 




Edgewood ISD 


3 




El Paso 


3 






Pt Worth 
ri« vyuiiii 


U 


District only 






U 


District only 




rlUuI'OuJl J U uJl^MJ&mO 


U 


District only 




5an Antonio 


3 


Virgiiiia 


Accomack County 


0 


District only 




Hampton City ' 


. 0 


District only 




Norfolk City 


0 


District only 




Roanoke County 


0 


District only 




Smyth County 


0 


District only • 
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Schools 



It was realized early in the study that the only means of preventing large gaps of data^Cas in preceding elementary 
school surveVs) about certain small programs and isolated pupil subpopulations would be to develop the capability of 
stratifying tke sample of schools^ Accordingly, a precanvass procedure was developed and pretested. This procedure and 
the results of the precanvass are described in appendix A. 

« Once the precanyass was completed and the data obtained'from it reduced, keypunched, and matched to the original 
school universe subfile, a sample of three elementary schools was selected for each full participation district. This 
sample of schools wasxhosen in the following manner: , . ^ 

* (1) Certain /schools that had high concentrations of target pupil subpopulations (migrant, bilingual, Indian, or 
ha^dicJpped) or programs funded by ESEA title I Migrant, ESEA title VII, or EHA, part B, wer^ included '^vith 
certainly" to ensure a large encjrugh subsample for pretesting fhe supplementary sampjing procedures. 

(2) ^<The regaining elementary schools within each district were oversampled wit6 probability proportional to size. 

(3) Depenkirig on how many sch^Jols had already been picked with certainty (in point J above )i the additi-onal 
■ numbir of schools needed in each district were drawn-^onr those in point 2. Special attention was gi*ven to 

choosing schools that had pupil target groilpjs and Federal programs of interest, ^ 

It was not p(4sible t^Jollow these procedures for one State because it coqipleted the precanvass task too late. Generally 
the following consi(|f^rations were taken into accotint while implementing these procedures: 

1 . Schools that had onfy prekindergartcn or kindergarten grades were excluded. 

2. Attempts were mades|o*^ick a sample of schools that was representative of the distribution of programs across 

schools within ^ach ^at^^ , • 

3. WhereV6r possible, atteir^ts were made to pick schools of varying characteristics with respect to the composition 
and size of the school population. . . 

By using these procedures, a sample of 101 schools. was chosen in 35 districts. 



Pl^ARATION FOR THE MAILOUT 



While the selection of participating^States and LEA's was taking place, RMC Was also working on the definitions for 
the questionnaires and survey support materials. Manuals were developed for each level oT participation (district, 
school, and teacher) as well as the forms necessary to carry out ^e procedures outlined in the manuals. The 
development of alternatives to be tested as part of the pretest resulted ilftvtJKo formS of the school manual and Pupil 
Code Sheet (PCS), in addition to alternate forms of each of the four questionnaires. A more detailed discussion of the 
survey support materials appears in the following chapter. 

In the m9antimc,'thc clearance process with 0MB was taking much longer than expected. The clearance package had 
been submitted by OE to 0MB ori January 1 1 , 1974. Since it was realized that delays in the mailout could have very 
serious negative effects on ihc entire pretest effort, it was decided to begin the design and layout work on the 
questionnaires prior to the receipt of clearance. However, all other composition and preparation, as well as printing, was . 
•held until 0MB clearance was received. RMC Was ijiptified of 0MB clearance on March 20. * " 

On April 12, I974,%irvey materials were ''ship|)pd to the SEA in eight States and directly to the fiye districts in 
* Florida. Districts were askcd'to return materials within 2 weeks. However, most districts were unable to comply with 
this* Request. In many States, the materials had arrived during the spring bf0ak«?ln one case, the State survey coordinator 
was away and the mat^ials remained in his office for a week. Several districts indicated that the CPIR and SSES arrived 
simultaneously and siifte they felt the CPIR had priority, completion olf the SSES was postponed. Several States had 
early school closings and stated that forms would have to be completed after the schools hdi closed. It became clear 
very early that the delay in 0MB cfearance and the resulting late shipment of survey materials was going to have serious 
effects on the pretest. - , , 

Tli^ contractbr established early con^ with the LEA's that were selected for the validation study site visits and 
asked them to hold their survey materials until after the site visits in May. When survey response was still lagging 



LBAtTmwIuJf ''^^ '""''^^ '° ^^^'^ "''^ ^ directl/to nonresponding 

MADLOUTfeUESPONSE RATES / 

The' following pages contain lists of when the returns for each- district were received it Intran Corporation in 
NKnneapolis. As mentioned above, the districts selected for the validity study held their pretest materials at the district 
office until, the site visit work was completed. Consequently, they are shown on a separate list. Tlieir resporise dates- 
should be evaluated in terms of the site visit period rather than the mailout date since they were asked to mail their 
questionnaires after the field staff had left their LEA. (T\vo districts in Kentucky were not included in the validity 
study subsample list because their returns were received prior to the notification tahold. These were both district-only 
participants.) ' . ' 

In addition to the listing of distriicts by date of rehirn. four figures are pressed that show: 

• Returns by Week-Validity Study Subsample 

• .Cumulative Percent Returns-Validity Study Subsample 

• Returns by Week-All Other LEA's - . 

• Cumulative Percent Refurns-All Other LEA's 

Figures 4 and 5 show two groups: the total of all (fistricts not tn the validity atudy subsample and a Subset of that 
>total showing the district-only group alone. ' / - 



Figure 2.~Returjis by week-Validity study subsamplc-SS^S pretest (spring 1974) 
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Figure 3. -Cumulative percent returns-validity study subsample-SSES pretest (spring 1974) 
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Figure 4.-Returns by feek-all other LEA's-SSES pretest (spring 1974) 
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' Hgure 5. -Cumulative percent returns-all other LEA*s-SSES pretesTCspring 1974) 
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District Returns 

DistHctsfncludedin the Validity Study (with' the exception^ of TYimble iE^unty, Kentucky, and Carroll County, Ken- 
tucky). The numbers below show school and teachier questionnaires returned as a fraction of questionnaires mailed out. 
All 19 districts fiere returned district questionnaires in 1974 on the dates shown. 





aiaie 


Date return 


School returns/ 


Teacher returns/ 






total schools 


total teachers 


Brevard Co. 


Fla. 


5/20 


3/3 


12/12 


St. Lucie Co. 


Fla. 


5/21 


3/3 




Collier Co. 


Fla. 


5/22 


3/3 


n/n 


El Paso 


Tex. 


5/28 


3/3 


g/12 


Brownsville 


Tex. 


' 6/4 


3/3 


12/12 


Grarfd Rapids 


Mich. 


6/6 


3/3 


9/12 


Holland City 


Mich. 


6/10 


3/3 


10/12 


Edgewood ISD 


Tex. 


6/17 


3/3 


* 11/12 


Somerton 


Ariz. 


6/17 


I/l ■ 


24/24 




Mich. 


Of 17 


5 jo 


fi /I o 
9/12 


Paterson 


N.J. 


6/19 


3/3 


12/12 


Boulder Valley 


Colo, 


6/24 


3/3 


17/52 


Atlantic City 


N.J. 


6/25 


3/3 


17/52 


Fayette Co. 


Ky. 


7/1 


3/3 


12/12 n 


Adams Co. 


Colo. 


7/2 


3/3 


12/12 ^ 


Wilson 007 


Ariz. 


'112 


J/2 


0/48 


Vincland 


N.J. 


7/5 


3/3 


7/12 


Nogales 


Ariz, 


, 7/9 


3/3 


32/52 


CQlorado Spgs. 


Colo. 


7/19 


3/3 


12/72 *" 


Total 






53/54 


288/516 



Districts not included in Validity Study, (Trimble County, Kentucky, and Carroll County, Kentucky, tfi^^tncluded here.) 
The numbers below show school and teacher questionnaires returned as a fraction of questionnaires mailed out. All 33 
districts here returned district questionnaires in 1974 on the dates shown-. 
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District 


State 


Date\return 


♦Trimble Co. 


Ky. 


\ 

4/2^^ 


•Carroll Co. 


Ky. 


4/25 \ 


San Antonio 


Tex. 


4/29 ^ 


•*Ft. Worth 


Tex. 


4/29 


Broward Co. 


Fla. 


5/1 


•Paintsville 


Ky. 


5/2 


*McAUen 


Tex. 


5/6 


♦Carlisle 


Ky. 


5/8 


Fulton Co. 


Ky. 


S/IO 


•Smyth Co. 


Va. 


5/15 


•Roanoke Co. 


Va. 


5/15 


•Norfolk City 


Va. 


' 5/15 


•Hampton City 


Va. 


5/15 


•Accomack Co. 


Va. 


5/15 


Alice 


Tex. 


5/22 
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School returns/ 
total schools 



3/3 
3/3 



3/3 



3/3 



Teacher returns/ 
total teachers 



12/12 
12/12 



31/32 



12/12 





V, 


Date return 


School returns/ 


Teacher returns/ 


District 

o 


State 


total schools 


total teachers 






5/28 


3/3 


12/12 




Mich.l 


6/3 


3/3 


12/12 




Kv \ 


6/3 






Somerset Co. 


Ky. 


6/10 


3/3 


12/12. 


Downe Twp- 


N.J. 


I 6/19 


0/2 


8/8 


Ft, Lupton 


Colo. 


1 6/24 


. 3/3 


10/32 


Dade Co. 


Fla. 


73 


3/3 


43/52 


*Minn, Spec. 


Minn. 


1 7/3 






♦Blue Earth 


Minn. 


• • 7/3 






♦Albert Lea 


Minn. 


1 7/3 






♦Roseville 


• Minn. 


i 7/11 






♦Hooldns 


Minn. 


1 7/11 






♦Moorhead 


Minn. 


7/11 






♦ Rre c ke n ri dee' 


Minn. 






■ - - 


♦Owatonna 


Minn. 


1/ 7/19 






♦N. St. Paul 


Minn. 


r 7/19 






♦Austin 


Tex. 


7/19 






Pointiac City 


Mich. 


1 7/19 


2/3 

r 


8/12 


Total 


j 

/ 




29/32 ■ 


172/208 



♦District-only participants. 



/ 

Summary: Pretest Return Statistics-Districts 
/ * Mailed Returned 



Validity Study subsample, 
AH others 

(district-only subsample) 
Total sample 



19 

42 
(26) 

61 



19 
'33 
(22) 

52 



Percent 

100 
79 
(85) 

85 



Of the 52 respondingv^istripts, all 52 returned completed district questionnaires. A breakdown of returns 
participating schools and teachers in those districts follows: 



Schools 



Teachers 



No* 


Percent 


No.* 


Percent 


53/54 


98 


288/516 


56 


29/32 


91 


172/208 


82 


^/86 


95 


460/724 


63 



Validity study 
All others 



♦The numbers In this column sho^SNschool and teacher questionnaires returned as a 
friction of questionnaires mailed ot 
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GENERAL OBSERVATIONS' 

0 

A brief glance at the statistics and graphs produced from the response to the pretest reveals the following points: 

1. The response curve for the validity study subsample doesn't tell miich of a story. It seems as though many should 
have taken much more time to return thiex materials had it not been for the followup phone calls. It would also 
seem that the phone calls directly to the LEA's (the latter part of the followup period) had considerably more 
impact than the calls' to the SEA's. o 

2. Returns for the validity study subsample accumulated to 100 percent.. This was possibly caused by two 
Jiappenings: \ . ^ * 

• an additional commitment was made to the pretest upon agreeing to the site visit, and 

• a few districts were persuaded to participate in the pretest when requested to be included in the site visits. 

The cumulative return line has a much sharper slope for the subsample. 

3. The response curve for all other districts shows a very definite (and more normal) picture. There are two definite 
peaks: the first- within th^ normally expected response time and the second showing the effects of the followup 
phone calls. 

4. ThQ district-only subsample shows a definitely improved initial response, but the, curve also dips just as quickly 
and flattens out for just as-long as the total group. The effect, then, of not having the schools involved was rather 

^ short lived. In addition, the overall response rate for the district-only LEA's was only the same as the total'pretest 
sample. ' 

5. It would seem, then, that the district survey coordinator is the key to a successful survey: 

• District-only returns were generally no better than all-other groups. 

• The response picked up once the district. was called, whereas the calls to the State appeared to have had little 
effect. 

• The return rate for the schools imthe districts that made returns was excellent. 

6. One final observation is warranted: the percent of returns for participating teachers was much lower in the 
validity study subsamgic because of the fact that all but two of the supplementary sampling schools were in the 
subsample. Teacher return rates in supplementary sampling schools were quite low as a group. 




1 



Chapter 4 , 
LOGISTICS • « 

INTRODUCTION 

A major facet of the pretest of the SSES was the actual operation of the survey-often referred to as logistics. 
Logistics is a rather broad concept and covers many different operations of a survey that take place before, during, and 
after the survey. Since NCES requested that less emphasis be put on logistics than had beeiflhe initial intent, it was not 
possible to actually pretest all of the SSES operations. However, attempts were made to conduct as many aspects of the 
pretest as possible in much the same way a full-scale survey would have operated. This chapter addresses those aspects 
of pretest operations where adequate evidence exists for making observations. 

This report on logistics is divided into two parts. The first part is a description of what was done and the results that 
were seen. The second part takes those results and relates them to recommendations foc^Jthe full-scale survey. In both 
sections, the general* areas to be covered include: survey support instrumentation, shipping and distribution methods, 
the survey management syslerti, and the processing of suiVey returns. 

PRETEST LOGISTICS RESULTS 

Survey Support Instnimcintation f > 

• ^ % - 

. Instrumentation, ^ \ 

A survey the size and scope of the SSES obviously could not be carried off with questionnaires alorie. There had to 
be other materials to explain how to properly complete the survey and the questionnaires, to explain who should be 
respondents and how to select them, and to supply supplementary information needed to answer somc^parts of the 
questionnaires. The materials developed for^he pretest to serve these purposes were: 

• various letters to State representatives, district survey coordinators, school principals, and teach(?rs; 

• district, school, and teacher manuals; 

• sample selection forms; 

• the Record of Teacher Packet Distribution; ' ' 

• the Pupil Code Sheet (PCS); 

• a form for districts to use in the return of completed survey matorials;.and 

• the district and school lists of special programs. 

All of these Materials were developed after the survey questionnaires were completed. Some of these materials could 
not be worked on ^until the design of the entire project was complete and until it was certain thtit there would be no 
further changes in the questionnaires or project design. This was particularly true of portions of the manuals and of the 
lists of special programs. Other forms, such as those developed to implement thd alternative sampling procedures, were 
independent of the content of the questionnaires. 

Because of the decreased emphasis on the -logistical aspects of the survey (the feeling expressed by NCES bping that 
the testing of the questionnaire and sampling alternatives deserved considerably more emphasis^, materials were 
developed to serve a particular need, but no alternate forms or procedures were developed for testing. Tlic alternate 
forms and procedures created for the two basic sampling* plans were developed to test two separate sampling plans 
rather than ta provide alternatives for logistics. Therefore, the results of the survey instrumentation will be rather 
simply stated. 
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Results. 

Ip general, the survey participants apparently h^d little trouble with the pretest in tergis of understanding what they 
had to do and'how to do it. Rather, the basic problems expressed to the v^dators were the ability to provide the data 
required and the time required to Complete the survey. " ^ ' '^'^ • 

• . Survey Communications, All of the contractor and NCES contact prior to, and during, the survey was with the SEA 
representatives. Oriljrone of the nine States indicated tliat they thought the presurvey communication was inadequate. 
However, of the li dfstrjcts t)iat responded to the question, 7 (over 40 petcent) indicated that« they would liked to have 
known more abomrjthe survey beforehand. Considering that some of the districts may hav0 felt communications were' 
adequate because they were also contacted by RMC prior to the survey about the ^te visits, there seems to be an 
indication here that the State coordinators do not pass enough iWonnation along to participai^ 

The letters for the district coordinator, the school principal, and the sampled teachers were included with the survey 
materials more as a courtesy than as a means of providing real cortimunication.'It Would be possible, but not always 
practical, to send letters to participating districts and scjiools i^ri^r to the survey. 

Manuals.^The manuals were the core of the explanation of the survey intent and procedures. They provided general 
backgrounli information, administrative procedures, sample selection instructions, and general instructions for. 
answering the questionnaires. Sipce little is known about any major survey administration problems, it is assumedjhat 
they generally served thei^ purposes wdl. However, based on a review of the comments from some validators regarding 
problems with certam items and^asa result of the item analysis, it is felt that certain items should be explained in ^nore 
detail than wpuld be practical to do on the questionnaires themselves. * ^ " • 

* Sampling.Forms. The various sampling forms evolved from* the development of the thrge basic sampling procedures 
within the schools. The results of using these forms can be stated only in terms of evaluating the sampling procedures 
themselves. Chapter 6 presents a report of these results. , 

Record of Teacher Pacf^t Distribution. An initial Ipok at the forms used in the pretest would lead one to think that 
this form w^ts. used for sample selectipn. It was commoyn to both sampling alternatives and appeared ks a place to- record 
the results of the classroom sampliflg procedure. , • ^ ^\ ■ " 

Actually, the^orin also served other purposes. It was intended to risiitind the principal about the teacher survey 
packetis 2^d to provide a record of wKich teachers were selected, the daie they received the sufvey materials, and the 
date each teacher returned the completed materials. ^ ' ^ 

Tfiijs form is not an integral part of the survey operations. For instance, the results of the classroom sampling could 
be recorded on another form. However^ it is felt that any additional help that c£(n be provided to the principals -will in 
the long run help the overall survey efficiency. While the overall response of completed teacher questionnaires- in the 
pretest was pQjOr compareH with the school and district questionnaires, this shoiild riot be interpreted to mean that^' 
additional controls are needed for the within-schook administration. The overwhelming majority ofi missing teacher 
questionnaire data was from schools selected-to use the supplementary sampling procedures. I^s recommended 
elsewhere in the report'that the supplementary sampling" procedures be dropped from ftirther-cfonsideration., ^ - 

Pupil Code Sheet (PCS). The PCS is designed to serve a number of purposes. It provides a place for the teacher to 
record the results of her pupiT'sample selectori, it serves as a place to record the pupil name for reference without 
placing the pupil name on- the questionnaire, and it serves as a record for postsurvey validation. In addition, for the ^ 
pretest only, the PCS indicated to the teacher Jhe number of pupils she should sample from her class. 

There are no indications to show that the^PCS provided any problems, that it did not serve its purpose,. and that it- 
should not be used in its present form for the full-scale surA^ey. ; ^ 

'''Return of Materials" Form. Aifter the district had gathered all the completed survey materials from the schools, it 
was faced with the problem of returning the materials to the contractor or to the SEA. To pjovidexontinuity in the 
survey operations and a means of diminishing and tracking nonresponses, the districts were provided with a form to 
record conipletion of survey operations and the shipmei^pf completed materials. The form was sent to the SEA 
indicating the date of completion and shipment and wheMr the materials were shipped to the contraptor or to the 
SEA. . . - - . ^ . . • ^ ^ ■ , 

. ■• • ^ 27 .^-^ ^ . . 
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No'negative feedback was received ortth^ use of this fom^^ • ' • 

District and School Lists of Special Programs. These lists of special programs were provided as a means of giving 
schools and teachers information necessary for providing some of the answers requested in the questionnaires. 

The district list is designed for the LEA survey coordinator to indicate the des^nption of the federally funded 
programs operating in the district. This description consists of the name of the program, the grades (prekindergarten 
through 6) that It serves, and the Federal funding source. The district was instructed t6^ make copies of this form and to 
give one to the principal in each of the sampled schook. * . * ; 

Each principal would, then use this information to complete Ue school list, including only those programs;, that ^ 
operated in the school. Copies were then to be made and distributed to each sampled teacher. 

Certain problems with the list were uncovered: 



It was often difficult to kndw how to define and name n program-whether t6 Ust the ESEA tjtle I Migrant 
program^for instance, by all of' the various areas it covered or to list it only once..The former procedure can be 
very tiiii'e consuming in some cases, while the lattejjr procedure is subject to the deficiency of not getting enough 
information to the peppld^who need it. v ^ " . 

A similar problem arbse where, say, '^ESEA title I Migrant" was Usted by the district office but the program was 
better known by anot ler name in the schooH, such as the "Migrant Foiod Project. " 

(3) It was often^ difficul: for a principal to know for sure whether or not each program listed for the district 
. -operated in his school .'in the same /manner, it was very difficult for a teacher to know for sure if the programs 
' listed for her school dj )erated in her class or whether the sample'd pupils were served by them. 

(4) Funding sources for ^deral programs were used that had Uttle bearing on the survey as a whole. This'served to 



(1) 



(2) 



confuse things somew! 



(5) Copies of the completsd lists had to be made, which may have been an*annoyance to some. 



Shipping and Distribution M^ods 

/ 

Introdt^ctiori. • 



at for some respondents. 



Prior to setting up the survey operations for the pretest and while the States were being soUcited for participation. 



f\,J vr^wluVAvr.w ' J ^ ' * 

each State was queried as t^ whether they^ would allow the survey materials to be shipped. directly to the sampled 
.districts or whether they war ted all of the materials to be distributed through the SEA office. All of the States except 
Florida indicated a preference for handUng the distribution of survey jnaterlals to the districts themselves. 
' The Wantage for the SEi^'s in allowing direct district shipment was two fold: they would not haye to receive and 
then redistribuie each box, a id it would save them money since tiiey would not have to pay postage or freight changes. 
The advantage to the survey operational was that considerable time would be saved in the distribution process. One 
potential disadvantage to thel survey as a whole was that shipping costs would be h^hefr following the direct district 

methad. \ ^ 

The promise of not distributuig materials to the districts until the State representative Had seen information copies 
of all materials seemed to have iitiie effect on their decisions., 



Pretest Procedures. . , . ' 

The packaging procedures were set up for shipments to go directly to'the LEA's in Florida and to ihe SEA in each of 
other States. All district materials for a given- State were sent at the same tune in one shipmenf. Each district 
represented one .box and each box had two labels: one for the destination SEA and the other to identify for which 
district the enclosed materials were intended. . ' * * , , 

At the same time the shipme;its were made, each .State was also sent a special delivery package cdntaining 
"information copies of all materials and a letter indicating fhe date of the shipment and the^ number of boxes* in the 
shipment. Enclosed with the letter was a return postcard on which the State surveiy coordinator could record the date 
of shipment receipt and the number of boxes received. The State survey corordinators were instructed to mail the 
postcard as'soon as- the shipment Vas received. In addition, each district packet ^in the top of the box)hada business 
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reply postcard attached on top. In the same*^ manner, a postcard was supplied for each school packet. Instructions, 
indicated that the.postcards should be returned as soon as the respective shipments were received. ' 

The proceflures followed for Florida were similar, but yaried so'mewhat because of the direct district shipments. The 
SEA, wa^ sent a package of informational materials and a lej^er indicating that shipments had been made to the selected 
districts. Each district received a letter under separate -cover and the return postcard was included with this letter. 
Beyond this, the other procedures were identical. All Shipments to the Florida districts were made on tJie san\e day as., 
the other State shipments. 

Return Postcards ^ lo . / * ''n^ . ' 

. States. Six of the eight States returned their postcards. Five of these SitStes returned'them when the shipment was 
received, whei;eas the sixth waited^ until the following week. As far as can be determined, there was no delay in the 
receipt of this shipment, so it is felt that the postcard was not mailed ri^t away. 

One way to make sure tl^ere would be no delay in notifying the addressee that the shipment has been received in the 
SEA ^building would be to provide for inside delivery, to the addressee. However, it is unlikely that this would be 
desirable for the fulL-scale survey since most of the sKipments to the SEA's will be very large. / ^ 

Districts. Postcards were attached to district packets only for the full participation districts. This would mean, then, 
that they were sent to the 5 districts in Florida and to 30. districts in six other States. Figute 6 shows the cumulative 
percent return of these. postcards over the weeks following the shipments. (Florida is graphed separately.) 

Altogether, postcards were received from 4 of 5 district$ in Florida and from 2apf the other 30 districts. In addition 
to the Higher percentage of returns for Florida, it.should be noted how muphTmore efficiently the postcard system 
operated for Florida districts than in the districts of othei;. States in terms df tl]e time it took for tjie postcards to be 
Feturned. This is presumably due to two factors: 0) the districts received the shipments faster than those in other 
States, and (2) more attention was drawn to the postcard because it was enclosed in the letter. 

Schools, Postcards were attached to the school packets for 101 schools. Of these, 15 schools were in Florida. 
Postcards were received from 14 of the 15 schools in Florida and from 46 of the 8.6 schools in the other States. Figure 
7 sho\ys the cumulative percent of returned postcards, with^ Florida shown separately.. Once again, not only. does 
Florida sho>y a much tiigher rate of return than other Statesj but ajso the postcards were returned faster from Florida. 
Thus, the^survey monitoring system seems to operate best with the direct district sliipment procedure. ' - 

One point noticed in tallying the returns of the school cards was that the postcards from all three schools in a given 
district wer6 almost always received during the same week. Any delays, then, were probably caused by the district 
officer rather th^n by the school principals..-- * . 

► Results of Interviews. ^ ^ . ' • 

As part of the site Visit procedures, interviews were conducted with each of the nine State representatives and with 
1 7 district representatives in the 20 districts viSited. Part of these interviews had to do with the logistics aspects of the 
survey. Responses to some of the key questions are as follows: 

. A. State^ Interview Responses * 

L Do you prefer to have the SSES shipments come through your office? . 
Yes = 6 No = 3 

2. Did you open each box? ' - * . 

- ^ Yes = 2 No = 5 

a. If YES: then, do you thjnk it is necessary to label the boxes as to which district thgy are intended? 
Yes = 1 No = 1 
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b. If NO*: do you, then, think it is necessary to ship all the materials first to your office? 
Yes = 6 « ^ No = 2 (Total respondents should have been 5) " 

M YES (to m: would it possibly be satisfactory to ship them directly to the districts if you had first 
received your Information cbpies along with a roster of participating districts and schools? 



Yes = 2 



No = 3 



3. For the full-scale survey, would you possibly call a meeting of the survey coordinators of the participating 
districts (as some States\do)? ' 



Yes=3 



\ 

Nd = 6 



4. (From the infomiatidn provided by seven of the interviewees, an average of 2.5 days elapsed between rec^pt ^ 
% of materials and forwarding to the districts.) How djd the materials get to the districts? 

* 3: First class mail.. ' 
2: Hand delivered by State personnel . . ^ 

2: Parcel post - . • 

1: Shipped by truck 

, ^ ^s- _ " ■ . 

. 5. Do you recall whether or not you returned tjie postcard as soon as you received the materials? 



Yes = 4 



No = 2 



6. Do you prefer to write the letter to egch participating district, or do you think it would suffice to have a 
general printed letter in the district packet? , 

" \ ' ■ ■ '■ . ' • 

)S = letter from State person 

2\ general printed letter , 
(Florida was excluded from this question.) 

-ft 

If the latter, this would mean, then, that the district coordinator would call the contractor, rather than you, 
'With any questions. Would this proccduic ue okay? 



Yes = 3 



No = 0 



B. A Quick Look at the State Responses ^ 

K The majority of the States prefer 4o have thb shipnnents come through thier offices. However, three States 
indicated a preference for not doing so. Five of sfeven respondents indicated' that they did not open the box!es. 
ii is difficult to understand why six i^tates prefer'to have the materials come to their offices when four of 
them did not 6ven open the boxes to inspect the contents. 

Of the two. States that opened the boxes, one felt that the boxes should bp labejed^^ith thp district name. 
The only purpose for putting this label cin the box was to eliminate the need for opening the box to find out 
for whiqh district it was intended. The response to question 2b is generally consistent with question 1. Since 
too many answered this question, it is difficult to compare the responses to que;3tion 2b with question 2. 

However, response to the next question shows that two additional States indicated they would consider 
direct district shipments under the conditions 'cited, (These arc* presumably not the same, as the "NO" 
respondents to question 1, since we show five respondents total, following six "YES" respondents to 2b. J 
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2. One State^ evidently cSSled a meeting of representatives from each participating district, while three indicated 
that they would probably do so for the full-scale survey. ^ * 

3. It should be noted that 'the incident of use of First class mail is probalily causied by sending small packages for 
the district-only participants. Since the volume of materials for the, pretest is so much smaller than for the 

' • ' full-scale survey, it is unlikel5^ that the responses here really indicjife jj^ything meaningful. 

4. Two States indicated that they did not send the postcard, right away. However, only one State postcard was, 
received late. Presumably the two nonrespondents to this question are the -same as the two who did not return 
the postcards at all. Evidently the jState survey coordinators did not mind sending the letters to the districts. 
It is unfortunate that there was nota series of questifuis to the effect; Did you use the model letter provided? 
Do you endorse this concept, or'prefer lo cQmpose your own? Did writing the letters hold up shipmen t ty. th e 
districts? Did you send tbe letters at the same time as the materials? . 

In additioif to the two States that felt a general printed letter would suffice, one other State evidently^ 
preferred the idea of having the districts deal with the contractor for questions about the survey rather tha(n 
dealing with the State survey coordinator. 

C. District Interview Responses 

1 . Did you have any difficulty with the way materials were shipped? 
ot clearffor whome, or what, intended. 

ethod of shipment was not the best to get to me. What way would bje better? 'None \ 
recommended. One noteworthy cqmment'^lis-Uiat the notation on the bo\ "Box 3 of 4" caused 
some confusion. For example, one LEA said thiu it *Vas waiting awhile for the other three 
boxes." (This notatioTi was caused by the shipments to tfie SEA's. Standiird shipping practice calls •* 
for labeling the four boxes in. the shipment "1 of 4," **2 of 4," etc.) This problem would not occur 
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if shipments were made directly to the districts. 
Other (Specify)f 

1 = received late (by mail) 

1 = no letter of transmittal iosido * 
1 = delivered by "grapeyine" from SEA • ^ 

= well done , . 

= should box edch school separately ^ • » 

' Other comments not pertinent » * * ^ 

2. Did you find the communications ade^ua^elfpi^or to the receipt of survey materials)? 

Yes= 10 No=7 ' . * 

3. Did you receive a letter from your State Survey Coordinator (excludifig Florida)? 

^ * \ ■ 

•■ ' Yes= 15 , No=2 

If YES: .. ; . . > • ^- ' 

' 9 = prior to receipt of material? 
4 = with materijfls : 
2 - aftpr receipt of materials ^ 

^ - IPNO: did you know who your State Survey Coordinator was? 
Yes= 13 / N6=^ " ^ 

' ' •28 
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4a. Do you remember if you returned the pdstcard as soon as you received the Survey materials? 

11= Yes, right av/ay * 

2 = Yes, but delayed opening the biox 

4 = Did not return postcard . . 

4b. Would you have opened the box and returned the postcard sooner if there had been a legend on the outside 
' of the box that.said: "Open Box N&, 1 and return the enclosed postcard IMMEDIATELY upon receipt?" 

Yes = 7 No = 8 ^ .^ 

"^a. Did you send the materials to the principals of selected schools, or call them to a meeting: ^ 

5 = sent materials * 

12 = called meeting 

^ ^ - * 

5b. Would it be better to call such a meeting for the Cull-scale survey -particularly since there will probably be 
more than three schools sampled in your district? 

Yes = 6 No = 3 . ' 

* 6. (From the information provided by 14 of the respondents, an average of 2.5 days elapsed before giving or 
' sending materials to the schools. Two other respondents indicated 28 and Sadays, respectively. Adding these 
two brings the average to 5.9 days.) How did you send the materials to the^schoojs? 

13 = hand delivered by school district staff 

1 = via school district vehicle or truck ^ . ' ^ 

3 = picked up by principals ' *> 

D. A Quick Look at District Responses ^ 

Somp of the interesling things that can be learned from the^e^esponses are that: 

(1) The districts often feit that they should have been told more prior to the receipt of survey materials-7 out 
of 17 thought that communications were inadequate. Only 1 5 received letter from the State coordinator and 
only 9 of these were received prior to the materials. 

(2) Not many of the districts bothered to return the postcards; the majority admitted that calling their attention 
to the card by the legend on the box would not cause them to act any more quickly. 

(3) The districts evidently find meetings preferable and it seems as though they arc quite practical about the most 
efficienrmanner of distributing",the materials. - ' • 




Survey Management System . 
Initial Specifications. "a . 

A good part of what goes into the Jogistical aspects of a large survey such as the SSES is some very affirmative 
planning. Such planning-of the pieces involvedMn the survay, of' the operations involved to make these pieces fit 
tbgeticr^properly, and of a schedule for these operations to happen on time and in the proper sequence -usually takes 
the form of a detaUed set of operational, specifications and schedules. Since \he pretest was considerably smaller in 
volume than a full-scale survey would be, the procedures were a reduced vei4ion of what would be needed for the 
full scale implementation of the SSES. 

In general, the first stage of specifications for the survey management system requires^ 
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■i (1 ) A general overview to provide orientation for the reader. 

. (2) A list of alt the materials that need to be produced or acquired-either as part of the survey package Or as needed 
for packing and shipping operations. 

(3) Specification of quantities: the quantity to be produced of each piece, the expected number that will actually 
be shipped, the number of different types of shipments, etc. 

(4) Detailed specifications of how materials should be collated and ass^hibled into sets, how sets should be 
packaged, and how to organize all of these materials for efficient operation. 

(5) Specifications for the development of the computer system that will take the sampled districts and schools file 
and produce: 

• labels for questionnaires, packets, and boxes-in the pfoper order and quantities for efficiency in the labeling 
and packaging operations; 

^ a roster of summary counts that provide an overview for labeling, packaging, and shipping operations; 

• detailed rosters that will be used for recording the completion of various steps in the survey operations and * 
allow for maintenance of up-to-date monitoring; 

• detailed rosters that will allow for. inclusion of a one-page summary for each district in the district shipmeiU; 

• punched-card tub files for use iaJceeping track of all returns and producing followup reports; 

• followup reports to be sent to nonrespondents for various mail and telephone followups; and 
© project monitoring status reports for various stages of operations. 

/implementation for the Pretest. 

Most of these procedures were followed for the pretest. The principal steps omitted were th'e use of tub files to 
rtionitor returns and the production of status reports. Because the survey volume fot the pretest was relatively small, all 
returns Were checked in by hand and status reports were produced on demand with little difficulty. 

Responses on both the return postcards arid the survey materials themselves were Jogged in daily and/or weekly, 
depending on the flow. These figures were used to produce the response tables in this report. 

In general, little difficulty was" experienced with survey management during the pretest. It is very difficult to 
properly report on how it all went or to evaluate the system since, on the whole, everything worked well and there were 
no alternatives to. be compared. Another factor that makes it difficult to evaluate how well the survey management 
system worked during the pFetest >yas that the'-analysis of alternatives concept introduced a great deal ofVariability into 
the materials production and asscrilbly. Complications were introduced that .will automatically be removed from the 
full-scale procedures. U is felt, however, that enough has been learned at various stages so that the, procedures used for 
the pretest can be refined for the full-scale survey. 

Processing of Survey Returns ' ^ 



Introduction. 

Tliis general category takes into account a number of operations: 

(1) checking in postcard returns to monitor shipping and receipt activities, 

(2) producing status reports to alert project management of potential problems on these distribution activities, 

(3) checking in questionnaire returns, 

(4) producing status reports on returns to monitpr receipts and to signal necessity for followup rosters, 
* (5) capturing data (keypunching and scanning), 

(6) two-stage editing and edit followups, and ^ 

. (7) editing resolutions and producing a clean tape. • ^ 

As indicated in the preceding section, not all of these pretest operations would allow direct reporting of results, 
evaluating results or alternatives, or comparing the results with what would probably occur on the full-scale survey. 
As a result of the small survey volume, operations 1 through 4 (above) were performed man|4(»lly. No tub tiles were 
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used for pulling receipts and updating the master file. Status reports^ere compiled whenever necessary without 
' computor assistance. All foUo3\$mps were done by phone; cpnSequer^l/no computerized followup system was developed 
or used. Data capture was done entirely by keypunching will full Key verification. 
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Frequency Distributions, 

>>-'■' 

Since response to the mailout survey was significantly delayed, it was necessary to cut off inclusion of responses for 
processing the survey data. All responses received through July 19, 1974, were keypunched. Tl^e total ntiftiber of 
•questionnaires received by that time was: 

\ ^ ■ ■ ■ 

• District questionnaire: \ 52 

• School questionnaire: \8l 

• Teacher questionnaire: 44\^ 

• Pupil questionnaire: 1,104 ""v^ 

Programing was completed at that time for running frequency distribution^ of the response^n these mailout survey 
questionnaires. These distributions (sometimes^ referred tp^ as marginals or univariate stffUstics) ipcluded counts and 
percents for each category response for each variable. Where items were not categorical variables, that is, they were 
write-in items, roponse intervals were preset for each item (or item part) prior to the programing for |he frequencies. 

The frequency distributions playpd a strong role in the analysis of survey responses and in the evaluation of each 
pretested questionnaire item. 

Edit Specifications, . . 

Detailed edit specifications were developed to check the logical consistency and accuracy of data provided on the 
questionnaires. In general, the kinds of checks that were specified can be categorized as follows: 

• Sum of parts;exceeds total provided in response to another question (base number) 

• Response directly contradictsjesponse provided dsewhertf ? 

— directly conflicting data 

- did not follow skip patterns ^ . . 
RECOMMENDATIONS FOR FULL-SCALE SURVEY LOGISTICS 

As stated previously, the logistical aspects of a nationwide survey arc extremely important -especially for a survev of 
the size and scope that the full-scale SSES is likely to be. Tlie outcome of the survey could suffer significantly without 
the implementation of a comprehensive and detailed system for the overall management of the survey. Without this 
system, the survey could result in incomplete or inaccurate data, delayed response to the survey, or overall poor 
response rates. * 

The recommendations that follow in this section are largely based on the results of the logistical aspects of the 
pretest that were reported earlier in this chapter. In some instances, the correspondence bdtween results and 
recommendations will be quite obvious. In other cases, the recommendations will be based on sources such as the 
comments recorded at the time of interviews with State and district personnel during the pretest, on the observations of 
the RMC project staff who participated in the site visits, and on the writer's experiences with the 1969, 1970, and 1971 
elementary school surveys. 

The areas that will be covered are: survey communications, instrumentation, and the survey management system. 
Survey Communications ^ 

Tried and proven survey methods have shown that the more communication that exists with survey participants 
before, during, and even after the survey, the more likelihood there will be of a successful data-collection effort. 
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' Communication pjrior to the actual mailout or shipment of materials is particularly significant because it call have a 
pervading influence over the entire survey effort. Good channels of communication during the survey ar^ ifnportant for 
solving logistical and substantive problems. Postsurvey communications (or feecjback) show the^^ts of the earlier 
communication in establishing good relationships and creating an atimsphere that will coe($urage cooperation for 
-^^ubsequent data-collection efforts. 

All three of these areas have direct application for the SSE^-BecauseHnis survey depends on close working - 
relationships and communication among various iQvels of respondents^ it is important for presurvey communications to 

'^egin as early as possible. Since the SSES is a very detailed and complex survey effort, it is likely that questions will 
arise during the course of the data collection. Effective communication links will, in most cases, easily splve these 
problems and wiU potentially increase the response rate and reduce the amount of foUowup work. In most instan^Tes, 
future NCES data-collection efforts will be handled by the same SEA representative and, in many very important cases,^ 
by the same LEA representative. In any case, it is desirable to have communications that are as intensive and as 
complete as possible to enhance the understanding of the survey participants and-taiielp eliminate anylpQtiaTtial^iming 
problems. 

The following is an outline of procedures that are recommended for implementation jn the full-scale survey to 
establish and maintain good lelationships with the^urvey coordinators in t)TCLl3tates.and to guarantee the success of the 
SSES. " • 



Presurvey. t 

, 

(1) Send an informational letter -to each State i^fcyfming it of the current pfeny and intentions \fegarding the 

full-scale SSES. This should be done 6 to 8 months prior to the expected survey start date. 

(2) Select a national LEA ssimple, excluding those States which absolutely refuse to participate. 

(3) Five to six months prior to survey start date, send a letter to each State announcing the LEA precan>?^ss effort 
and include a list of the sampled LEA*s for each State. States should be requested at this time to notify each 
sampled LEA (preferably by letter) of the impending precanvass data request. If States are going to be given an 
option to have all survey materials shipped to them or to have them shipped directly to the districts, they should 
be given the opportunity at this time to make such a choice. 

(4) When the precanvass materials are shipped to the SEA's, include a letter of explanation for each district. This 
letter should summarize the precanvass procedures, suggest distribution and followUp procedures for the 
precanvass, and a second list of the sampled LEA's should be included. The precanvass should be done as early 
as possible during the school year-probably about 5 months prior^o the survey. SEA's and LEA's should be 
given a number to call in Washington, D.C. (preferably toll free), to have questions answerepl'^nd problems 
solved.. ^ 

(5) Send a reminder letter to each State about 2 weeks later. The districts involved should probably be listed again 
for convenience. 

(6) Approximately ] month after the precanvass mailout, foUowup calls should be made to the States. Permission to 
send a reminder letter to the districts that have not completed the precanvass Should also be obtained from the 
SEA at this time. (Some sort of personalized letter would bo preferred and more effective.) 

(7) Sometime after the precanvass has been completed, letters should be sent to the States giving them more detail 
about the survey effort-particularly an up-to-date schedule. The States should bQ made aware of NCES's * 
intentions regarding survey procedures about 2 months prior to the survey start date. 

(8) Approximately 2 weeks prior to mailout, a letter should be sent to each State. This letter will accompany a 
sample set of all survey materials as^ell as a relisting of the districts to be surveyed. . 

Communications During the Survey. 

(1) Each State should receive a letter of transmittal at the time the shipment is made telling them the date of 
shipment and the number of boxes. If shipments are made directly to the LEA's for a given State, thesQ letters 

' would go to the LEA's. In this case, the State representative should receive a courtesy letter indicating that the 
• shipments have been made to the district. 

(2) Districts should receive a letter of explanation before they receive the survey materials. Most States prefer to 
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seiid'such a letter themselves at the time they are distributing the materials to the districts, but a letter from 
NCES or the contractor would be more effective and timely. It is hoped that the States would have already 
communicated with the LEA's prior to the survey. However, it probably should be assumed that the LEA's will 
♦need some further briefing. * ^ \ 

(3) States an4 districts (apd preferably schools, too) should be provided with business reply rkurn postcards to 
notify receipt of materials. This will assist the contractor in spotting shipment problemSi or distribution 
bottlenecks. 

(4) A telephone number in Washington, D.C., should be/provided for SEA's and LEA's to call if aiW problems or 
questions arise. This number should be toll free if at all possible. It has been shown in the pasKthat such an 
arrangement Jias a significant effect on participant cooperation because they view "Washington" asVrying to do 
as much as possible to make their job on "the survey easier. 

(5) About 1 .month into the survey the S^A's should be asked to remind the participating districts of >hc survey 

* completion date. At this time permission should be obtained to followup the delinquent districts dir\ctly-by 
either meilor plione. 

(6) Periodic foitotJIifJiwill have to be made to nonrespaodent districts. The letters sent should be as personajjxed as* 
possible and shouldprl?Yidejnformation about what is being requested, including listing the matcpakihat have 

• not yet been returned. 

Postsuney. 

(1) Letters should be sent to participating districts after their materials have been returned. These should also be as 
personalized as possible, should acknowledge receipt of survey materials, and should thank the district for their 
participation. . • 

(2) biates should also be sent thank-you letters after survey completion. 

ftyMarty State\ and districts have indicated that they would like to receive feedback on survey results. Plans should 
be made to provide for such feedback on whatever level seems most appropriate. This is particularly important 
for those skeptical States and districts that are reluctant to participate because they feel no good has ever come 
from previou? surveys and view current efforts as a waste of their time. The feedback should be provided as soon 
as possible to show how effectively the survey can be run and to jnentally prepare the respondents for future ^ 
survey^^f forts. 

General. & . 

Results of other surveys have shown that using specialized mail services adds significantly to survey responxes. While 
' it is realized that the use of government franking privileges will realize a certain cost savings, nonetheless, it is 
recommended that stamped envelopes, special delivery, or certified mail be used when feasible. The use of these special 
services generally draws extra attention to the communication and results in a more effective survey. 

Instrumentation 

The concept of instrumentation, a vital aspect of a successful survey operation, involves not just the actual 
data-collection instruments (or questionnaires) but also the various pieces of survey Support materials such as the 
manuals^nd forms used for recording completion of various survey operations. Careful consideration should^ be given to 
the conceptualization and design of each piece. The survey questionnaires could be perfect in every way and still be 
part of aiTffiTsiretessfuNataKJollection-effort-Wit^ the proper support instrumentation, Recommendlitions for the 
approaches to be taken with survey instrumentation follow. 

General 

(1) Setting typo: It is recommended that an automated system such as a composer be used to prepare the copy for* 
all instmmentation, as opposed to a standard typewriter. This is recommended because of the inherent 
advantages of the composer such as use of italics, different weight faces for highlighting, and different sizes of 
type of headings and titles. 
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(2) Use ofcolor: It is recommendejd that all the instruments for eacTi rgsponse level be printed on the same color 
stock and that the color for e/ach level be different from that for the others. The use of bright colors helps 
respondents notice and recogriize the survey materials, an advantage that was realized during the pretest. It is 

, - good to colol^key the instniijients for a given level because the individual respondents can identify a particular 
color as something they should be concerned with. This also helps those processing the questionnaires to easily 
distinguish one questionnaire from another. Different colored inks should bc*considered on occasion where their 
o use WQulJ serve a significant purpose. 

(3) Instructions: The pretest further substantiated the need for complete, clear, and concise instructions. Because 
respondents tend to not read or follow them, instructions should be placed frequently throughout the 
instruments and always close to where Ihey are needed. Where appropriate and economical, they should be set 

off in a different type face and sometimesi printed in a different color of ink. 

. ' . t ■ 

The Questionnaires. 

The following points should be considered in approaching the laydut of the questionnaires-the single-most 
important instruments in the survey: * . ; ^ 

(1) Questionnaires should be laid out in the manner thaf is most appropriate for the mode of-data capture to be 
-employed, including the provision of appropriate codes for da'tairapture. 

(2) Provision should be made for ease of clerical coding and pre-edits, if appropriate. 

(3) The-Qverwlielming consideration in design should be tp lay out the questionnaires in a manner that will provide 
for the easiest and cl6arO$t response thus reducing the respondent burden. ^ 

(4) Provision should be made for a preceding system that will permanently identify the respondentahrough all 
stages of survey operations. 

(5) Words that need defining should be clearly identified as such each time they are used. The definitions themselves 
should be part of the questionnaires as was done for the pretest. The theory is that the closer the definitions are 
to the need, the more likely ft' is that theyll be used. ' . ■ J 

(6) Certain important instructions should be repeated on the cover of the questionnaire so that they will be read, 
but their number should be reduced from that found on the covers for the pretest. There was evidence that 

* some instructions were missed with some consistency. The fewer the instructions on the cover, the more 
attention will be given to those that remain. Ceftain instructions are so important and were missed so often iff 

« the pretest that they bear r(?peating on the top of each page of the questionnaire. Instructions printed inside the 
body of the questionnaires, particularl}^^ routmg instructions (skip patternsj, should be printed in red ink to 
avoid their being ignored as they often were during the pretest. 

(7) Questionnaires shpuld be set up for data capture in a way that will provide the most efficient and econoipical 
process. The type of cfeta collected on the district and school quQstionnaires is such that they would be most 
adaptable to some fonVi^of key entry. The volume projected for each questionnaire on the full-scale survey is 
large enough that thi<^i^ute would be the most economical and i^^us the recommended form of data qi^pture. 



However, with the volume piojected for the teacher and pupil questionnaires (approximately 60,000 questionjiaires 
between the two) if. Would be far more economical to use optical scanning (or optical mark reading) procedu 
these questionnaire. In their cu^ent (pretested) fonn, these questionnaires are easily adaptable to an optical sLrining 
format. The fey^ questions that did not currently. have categorical responses (i.e., closed end items) were put uKthe 
"open end^<ir "write-in numbers" for/n for tjie pretest solely to establish the proper ranges and intervals for closed en3\ 
rcsponscf^on the full-scale instruments. (It should be noted that precisiorupf data is not lost by converting write-in 
numbers to categorical responses. Initial checks with data users determined That the data acquired by this means was 
niofe than precise enough for the analytical use to which it would bQ subjected*) Other items provide for a write-in 
response in the "other" category. These items also were set up in this fashion for the pretest only to establish whether 
or not the categories provided for response were . adequate. The intent was to check the writtcn-in responses for items 
that had ^a high i;icidence of "other" to see whether anything was written in often enough to warrant additional 
response categories. " • 

^e teacher and pupil questionnaires clearly collect the kind of data most suited to scanning. And, as mentioned 
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above, there will be a considerable niimber of these questionnaires to be processed. Both of these characteristics servt 
to make scanning the most economical mode of capturing the data. But there are other overwhelming reasons for 
recommending scanning as the most appropriate data capture technique for these questionnaires. From a logistical 
point of view, the greatest advantage is the speed with which the questionnaires can be scanned. Nearly as soon as the 
questionnaires come in they are ready for the machine: the ID number on the questionnaires can be used to update the 
master file with these returns and the data is qui^y ready in a computer-processable form for computer edit checks. 
(This is of considerable advantage in the method of applying edit checks that is recommended further on in this 
chapter.) Another advantage from the operational point of view is that the scanner creates an easy-to-manage computer 
tape record that eliminates the error-prone and difficult-to-manage handling of large amounts of punched cards through 
various stages of processing. 

However, more importantly, scanning provides significantly more accurate data than any key-entry process. 
Statistics have shown that scanners can be used to capture data through the optical mark reading process considerably 
more reliably than data can be captured through the keypunch/key-verify process (approximately 99,5 percent 
reliability versus approximatcfy 97.5 percent reliability). 

Manuals, . . 

Manuals are a very necessary part of the SSES procedures. There is little question but that the administration of the 
survey is.complicated enough that it certainly can do with no less in the way oflnstructions than has been the case in 
the pretest and previous ESS*s. It is clear that the respondents need as much explanation of what they have to do as can 
be provided to them. There is^too much that needs to be explained by way of background and rationale for the survey 
for it ail to go into a letter. In addition, there are a number of details regarding survey administration that need to be 
explained somewhere: Twn of these manuals contain sampling procedures. And, the item analysis of the response to 
several questions showed clearly that they lieeded more adequate explanation and examples^ SUcH^helongs only in the 
manuals and certainly not in the questionnaires themselves. 

A suggested revision to procedures contained in . the manuals \yould be to specify in the school manual that the 
principal should have a' clerk in his office fill out the top portion of the PCS for each tdacher packet. This form wouW?- 
then be used to ensure the corrept distribution of the teacher packets by checking the PCS that the clerk would liiye 
clipped jgJLlije4op«f each packet. A further extension of this concept would be to have the clerk also copy the teacher 
code n^ber onto each pupil questionnaire in the packet. This procedure was used by one school during the pretest, 
and it provided a 'great deal of uniformity in the recording of this information. ^ 
•We recommend not only that the manuals be retained but also that fyrther advantage be taken of their existence. 

Sampling Forms, ^ 

'-^ 

The sampling^ forms used by the school principals were part of the/manuals, i.e., they were the back cov^r of e^ch 
school manual. This procedure seemed to work all righ^^n that no complnints were received. 

It will be necessary to use some kind of form (or forms) in the clpssroom sampling procedure to provide a uniform 
method of recording procefjvral steps followed and results that caoJ^yaudited in a postsurvey check.- However, thpre is 
no strong reason to include them as a detachable part of the manual, as opposed to including a separate fortti in the 
survey package, except for the fac\t that the principal will know exactly where to find the form. Tlicrc is a slight 
additional advantage in that following this procedure would mean one less piece of paper has to be kept track of in the 
package assembly process. Aslight disadvantage, however, miglil be that the principal^would neglect to detach the form 
from the manual and file it for future checks. ^ • , * 

At the teacher level, 'the only form used for sampling pupils wis the P^S on which |he results were -recorded. 
However, it is recommended in ighapter ^ that a new form be creatfthfor the pupil sampling: a form on which the 
teacher v?DuId record the alphabetic roster of pupils. The same considerations afeut incorporating this form into the 
teacher manual woUld apply as above. , < * 



Pupil CodeSheet (PCS), 

The PCS is a sampling form, but it also serves another purpose. To give as much consideration as possible to the 
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concept of cpiifldenti^ty of data, the pupil nnjne is fiot recorded onto the pupil questionnaire. And yet, it is necessary 
io know whicb pupil belongs to which pupil ID number at the time of the reliability and validity (R&V) study. The 
PCS meets this need. / ^ . 

The»PC$ also provides" a place to record the names of the pupils .sampled to be used for chiecking at the time of a 
postsuFvey audit^ Ad(litionally,^he teacher caif use this form for referring to the name of the pupil for' whom she is 
responding on«a given pupil questionnaire. The PCS might also serve one additional purpose. If the pupil sampling were 
to be done by^he principal's office for each sampled classroom, the PCS could be used for indicating what programs 
the pupils participate in. « 

For the pretqst the. PCS was a two-part *'rio-carbon'-required" form. The reason for this wasrto help ensure that the^ 
PCS would be available at the time of site visits by having copies of it in two places— tl].e school files and the district 
files. SincCi all the districts that were visited for the pretest were requested to kepp all materials at the district office 
until;^the site visit, it, is difficult to evaluate how well this procedure worked; i.e., whether or not copies resided in both 
the district and the school offices. Since the PCS is more important than just a place to record the sampling results (it is 
also needed for R&V of the pupil questionnaire), it may be worthwhile to continue the practice of using two-p^t paper 
in an attempt to ensure that a copy will be available sortiewhere. In any case, the instructions to hold the PCS and not 
return it to the contractor should be made as clear as possible. 

T ■ * ■ • 

Return of Materials Form. ^ . 



There have in the past been pro blems^ with the return of the materials from the district. In addition, there have been 
problems, with the State (as indicatfed when queried by the contractor on the followup) not knowing whether or hot the 
piaterials for. a given district had been returned. Finally, there have also been problems with lost shipments or wasted 
phone callsi The form that the districts -us^d to notify the States that the materials had been shipped served two 
purposes: it alerted the State survey coordinator to the fact that shipment was on its way to the SEA offices, or it 
notified the State* survey coordinator that^the materials had been shipped to the contractor. 

It is recommended that all return shipments be made from the districts directly to the contractor. There is little, if 
anything, to be gained from routing returns through the SEA since it most certainly adds to the total cost of return 
shipments, u^nduly delays return shipments to the contractor, causes the survey monitoring system tQ operate less 
efficiently, and results in needless followup on -district return of materials that are sitting in the SEA offices. The States 
should bevehcouraged to allow the districts to return ih€ix materials directly to the contractor. And, in fact, perhaps 
they should not be given any choice. 

The States, do, however, have a right to^want to monitor the returns from their own State. Consequently, we 
recommend that Ihe districts be supplied with a* postcard to notify the^ State coordinator that* piaterials have beeil' 
returned to the (Contractor .'In addition, because shipments can be delayed and, at times, can get lost, it would be useful 
to su()ply districts with a busine'ss reply return postcard that they would mail to the contractor on the same day as the 
shipment is made. This card should contain places for the date materials are mailed, number of boxes, method of 
forwarding, and the name of the shipping agency or company. ' ^ ^ 

. Postcards would be the most efficient form' of notification but orily if they are actually used. Some thought should 
be given to devising a* clever rneans of .brining the postcards to the immediate aftd obvious attention of the district 
person when the Shipment is being made. - ' . - » 

Li^s of Special Program s^ ' * ' \, 

The question X)f providing information to principals and teachers about the^unding sources of programs operating in 
their schools and classrooms has always been a problem for this stirvey. The District and Scl^ol Lists of Special 
Programs that were developed for the priest represent a refinement of the forms used in the past. Respondents who 
were interviewed generally indicated little problem with the use of these forms. Yet -the validation and item analysis 
procedures showed results for some items that indicate the information ^as still not transmitted properly: ^There is no 
question that it is necessary to make sure principals and teachers have some means of verifying the funding sources for 
the special programs in, their schools and classes. ^ / „ ■ 

We V^ould recommend .the following further refinements to these procedures: ' . 

.'S- *' ' ■ . - ^1 - . 



JC 



(1) Directions should be reviewed and fine^unedin an attempt to clarify the situation for the small percentage that 
wa^.cpjafused. 

(2rMore'snecifically, directions should be explicit about hoVto name the program. Examples should be given in 
the district nianual of how to indicate the name of each program that is the one most likely to be 
recognized -€V(en to the extent of breaking down one program (such as ESEA title I Migrant) into the 
components by which people know it on a day-to-day basis. 

(3) Time, could, be saved for the district survey coordinator by indicating that only ^programs operating in the 
.sampled schools need to be 4isted, " ' . - ' 

(4) Confusion could be further eliminated by dropping the listing of all programs that are not the tai^get prograins 
for the SSES. (Recommendation was macie elsewhere. to similarly revise questipns regarding Feder^ program 
participation in the district (pretest question 9) and school (pretest question l 5) questionnaires. The concept of 

. **nori-Federal programs" should also be clarified by directions and examples. 

(5) The district form procedures should be changed such that the district coordinator wouldr • 

• make the necessary number of copies of the completed list of programs, 

• wri^e: in the name of each sampled school cm the copies in tKe^space provided, and 

• check off (in the column provided) the programs that apply for each school. ' 

. ^ . • ' ■ • ' ■ 

.. Following these procedures would eliminate some of the^ confusion that arose concerning whether or not the 
programswereapplicable to "all schools" or to 'tthis school." „ , ' 

V ;, / ■ - . ' ■ 

(6) The School List of Special Programs "should be eliminated because there is evidence that teachers either did not 
use it or did not understand it. In additiion, it represents too much of a-pxoblem in identifying which programs 
are applicable to a given classroom. The school manual should be appropriately revised as follows: 

». • • 

• A section should be added to clarify exactly how the principal should use the copy of the District List of 
Special Programs he receive4. (For the pretest, half of the principals interviewed found it necessary to contact 
the district coordinator regarding the list.) • 

• Instructions should be added asking the principal to specify the names and funding sources of the progc^ 
which am)ly to each^^^pled classroom, the classroom feachfer, and to any pupils in that classroom (wJiere 
not anj^5>jritate for the class as a whole). A special place for the prinicpal to do this should appear on the 
front oPwe teacher questionnaire. . , 

(7) While the District List of Special Programs should be color-coordinated with the other district materials, the 
shade of the color should be as light as possible to minimise any copying problems. ' - 

11 ■ a f 

The Survey^Management System * * 

,That the full-scale SSES is a large survey project lhat needs tight management control goes without saying.. It is 
probably one of the largest nationwide survey -effo^ outside of the work that the Bureau of the Census is doing. 
, Certainly there are other suiVey projects, that gather data from a larger sample Or from more respondents than the 
SSES, but therd are few that ^. . 

• gather data for four levels of response, > * 
. • depend on a three-stage distribution system, and 

• i^eed so nlany different survey support materials because of the complex jiature of the survey materials and the 
data requested. . . 

\ The ESS-'s in the j[)ast Were notorious for their late shipments and lack of proper survey controls. Strong attempts 
were made for the 1971 ESS to significantly tighten up the survey control procedures. To the Unfits of our knowledge 
it Appears that they worked, at least for the initial stages of th^ project. It is known at least that the 1971 survey was 
printed", assembled, and shipped in record time andahat'the ESS shipment date was met for the first time-despite the 
usual delays in 0MB clearance. ' * ' 
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The only way in which this was possible was by a tight.survey management system. Quite often it is assumed ihai 
the use of the«word "system" au-tomatically impliescomputer system. Well; that is at least partially true for the type of 
system necessary for a large and 'complex. sufvpy effort such as the SSES. To successfully control and monitor such a 
survey j a system is needed that interrelates a computer, system ' and a detailed set of procedures for production 
personnel to follow during the course of the project effort., This system must cpver all aspects of the project from the 
design and implemei^tation of the survey materials and their printing, assembly^ and shipmenl, to the check-in of 
receipts,,foftQWups, ^d processing of survey responses. / , * 

Survey Preparation. . ^ 

' One thing that was clear in preparing for the 19-71 ,ESS remains clear for the SSES: the size and scope of the survey 
i> entirery-too large to keep track of it all without the assistance of the computer. But the computer and the' system it 
runs can only be as good as the original specifications'that are laid out and the information that is fed into if during the 
course of the survey. The. amount of interaction necessary, during every stage of the survey, betaveen the survey 
operations (the system of procedures for production personnel) and the computer system is quite high. ;Careful 
planning-prior to any implementat^n-<©f every stage is extrepiely important to optimize the computer system. It 
should be thought out and developed by the same persori or team that is designing the survey operations. 

Every step should be preplanned. This holds true for the design of every piece of survey materials-from the very ' 
important questionnaires to the smallei^t but integral survey support form. It can be quite amazing how much effect the 
content of one form has on another. The production bf materials should be coordinated in such a way that 

. V. • ". ^ 

• • the finali2^ation of survey materials should occur within the same week; » I 

^ • each piece should be available for ppoofreading as soon as the composition, layout, and keyJining arecompleted 
for each; " ^ ' , ^ 

. • thd* same person (or team) who is familiar with every aspect of th,e survey's complexities should review and proof 
each survey piece; . . ' , < 

• it is important that last-minute changes and refinements be made by one person in a coordinating position who 
. has intimate knowledge of all aspects of the project and is in constant touch with each aspect -either the 

contractor's project director or NCES's project technical monitor; 

as the pieces are reviewed and proofed, their interrelationship becomes clearer; vit^ changes in content are often 
^ introduced to-suryey support instruments at this time; this process should take place on a flow basis; 

• the sign-off on each should be accomplished as much as possible in the order in which it is'produced-d complete 
piece at a time-so that it can move onto production (printing and binding) in such a way that the production* 
workload can be evenly distributed'over time to eliminate last -minute jam-ups. 

The timeliness of these steps i^ extremely imp^Ortant. It is essential that I^ERT (Program Evaluation aiid Review 
Technique) procedures be utilized Mn establishing the schedule and morlitoring progress. The computer comes intp play 
through PERTihg of all important steps. Unless sufficient time is set aside, iu do this work in an efficient manner on a 
schedule spread over time, preparation and pro'duction of the large volume of niaterials necessary for the SSES cannot ' 
be done on time;, without the use of a PERT system. The only coiupromise to adequate scheduling or using PERT is a 
budget large enough to consume all the ineconomies generated by doipg too much in too little time in too unorganized 
a fashion. ^ • 

The assembly of materials-is the next major step: The assembly will be very complex and every step must be thought 
through- and preplanned. Careful thought must be given-to the order.in which the materials shoujd be inserted in the 
packets. With the large numbers of materials involved in the SSES, the logistics of pre-assembly must be carefully 
planne'd so that once assembly is begun it will flow smoothly through every step. Only an organization with experience 
in handling many different pieces, of materials (up to 20 for the SSES) in large ^volumes should attempt these 
operations. Every step of the assembly process should be closely observed and monitored by someone with intimate 
knowledge and understanding of the survey's inner workings. Only in this manner can time-consuming errors be avoided 
and potential problems foreseen. The SSES materials production, assembly,^ and shipping will fully occupy large 
printing and mailing facilities for a number of weeks and an operation of this complexity must b? closely observed and 
monitored by key project personnel. 
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The computer system comes back into play with tjie labeling and packaging steps. Computer listing should be 
produced that will provide an overview of each segment of these steps. Thenurribers of materials and packets for each 
level should be, summarized for each higher level so, that checks can be made at key step^. Detailed procedures will 
direct assembly and shipping personneF on correct interpretation and use of these^ listings, how to make checks and 
correct errors, and the fnanner to be used in recorciing successful completion of key steps. The computer system will 
also produce labels to be applied to each packet (actually to each questionnaire). It is extremely important that these 
labels are produced in an order that allows the most recently labeled" product to be placed on^top of the one precedirlg. 
In that way, the materi^alS can be processed through the fiext step in the same order and the order of the materials never 
has to be inverted.-The end result of this procedure is thaf all materials are in the proper sequence for boxing and 
shipping. Labels should also be interspersed in the proper quantity at the appropriate places in the label sequence. 

This assembly and ^pping process will take about a week. It sl^ould be closely monitored at each step. Part of this 
monitoring processes tl^e information >that is supplied each day^ on the shipments that day, the nurhber of boxeS'in 
each, and the carrier name md waybill hwmber. This information is recorded on a shipment master roster an^d is vital 
for the shipment notificaffon letters that ^tfC^mailed each nigjit durirfg the course of'the'shifipipg pro^'ss. This is the"-" 
Qrst step in the very necessary survey monitoring syjstem and is vitaUo the tight/Control that must be kept on a survey 
of this magnitude and complexity. " ' ] ' . • ^ 

Management During the Survey Process: ; 

Monitoring. Distribution. The interaction of detailed personnel procedures and the. computer system* continue during 
the course of the survey. As postSards are received from the States and districts that have received their shipments, an 
up-to-date picture of the status of the distribution process can be provided. The compiiter^system will have produced 
punched'Card tub files that will bemused to monitor receipts. As sobn as a shiplrient receipt is ackngwledged, a notation 
and date is made on the master roster for e^ich State and direcf district shipment. A punched card is pulled for each of 
these in addition to all other districts and schools.* This method allows for immediate production of a status report by 
running all the pulled cards against the master computer file t^ update the status of each. Periodic, checks Of these 
status reports wUl alert^ the project staff to snags in the distribution process so that conrectivc action can be taken 
before significant delays are realized*. * - ' 

Shipping and packaging methodlogy will play a significant part in the responsiveness of the monitoring system at this 
stage of the survey^. Given the results of the pretest and comments during the site visit interviews, the following are 
recommended: 

(1) Packaging: Each school should be a self-contained package inside a box of multiple schools for a district. Each 
" district should have its own box(es) that should be labeledas to thp intended district wfien shipments are made 

through the States. The top piece inside the first bo^x for, each district should be a summary sheet that will 
provide an overview of the contents. It should also attempt to diminish the impact of tjie large amount of 
materials by noting the sm.all numbers of questionnaires that are required Tor completion at each level. A roster 
of the sampled schools in each district should be included in each district^shipment to provide the district 
coordinator with a further overview and a tool for his own survey monitoring. ' 

(2) States should be encouraged to allo\v district shipments. The pretest results in Florida gave testimbny to the 
efficiency of this procedure. States may not, realize what a tremendous volume of materials they would be 
committing themselves to distribute and how much delay State distribution adds to the sifrvey process. Several 
LEA's (outside of Florida) expressed preference for direct shipment. The addresses of the LEA's should be 
checked for accuracy and completeness in whichever States agree to direct shipments. In the past the addresses 
on the Public Sc°hool Universe File were often inadequate. 

(3) The postcard system should be- employed to^low for the cjose monitoring of the distribution process that is 
. " nedessary. Direct communication with the districts prior to receipt of survey materials will help in Ihe process. 

As'4n the past, letters should be sent to the districts. Since States are not likely to get letters to the districts 
before the districts receive the survey materials from the States, it^Avould be preferable to have ^e contractor 
mail lette?si directly to all districts regardless of the distribution methodTised. 

(4) Provision*' should be» npiade for efficient 'return of survey materials from the districts. "^Packaging and 
• ♦ shipping procedures should provide for use of double bokes with the inside printed box pre-addressed to the 
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contractor. This wUl e?»s^ the job for the districts but stil! leaves the problem ofpaynxenffor the shipment-a 
problem .that has come up often in the past^ It is unlikely that the use of government franked envelopes would 
work. The first problem "is that materials would have to be mailed back directly to the government. The second 
problem is that/';at. even the,§,chbol level, ther^ are too many materials for one envelope. Each school will 
• probably be returning 21 questionnaires. Us6'of multijple eavelopte for each school would significantly add to 
. . the contractor's problems in controlling and monltoring^survey resporises. Perhaps districts coulti be directed to 
use Federal funds, such as BSEA title I funds, to pay for shipments. There have been indications of heavy (buf 
unauthorized) incidence of this procedure in the past, t/any case, wfe recommend that materials should not be 
. ' retui'ned through the SEA's because of the kdded delays.and costs. 

>.•.'- ... • ' ■ 

Monitoring Survey Returns. As survey returns are received from the districts, receipt should be logged on the master 
foster. As the contents of each package are checked in, the use of another mqxe detailed punched-card tub file will 
come into play to permanently record each receipt. A card should be pulled, for each district, school, gnd teacher 
questionnaire received. If it is desirable to monitor the number of pupil (g[uestionnaires,,procedures can be implemented 
for overpunching the card for each teacTier questionnaire. . , , ' I 

Using this procedure will allow for production of weekly^ computer status ^orts of receipts. Qose monitoring of 
-these reports will enable the project monitor and project director to decide the best timing for foUowups to 
nonrespondents. Followups should take the form of listing, by district, the number of materials, by school, that have 
been returned as of a certain date as well as the number of materials that are outstanding. These followups can be 
personalized by use of computer produced letters for each district. To be done most efficiently, the letters should be 
mailed directly to the delinquent districts. A summary for each State can be given to the State survey coordinator since 
providing followup listings (by sc|;iool.and district) for the entire State proved to be highly ineffective in the past. 
Reports summarizing the entire foUpwUp activity could be automatically produced for project monitoring. 

Certainly one and perhaps two olr^more followups will be necessary, They should be performed in a timely fashion 
"that will be efficient and effective in keeping the overall survey response period down to t^he bare minimum. At some 
point toward the end of the r^gspcmse^ period, it will be necessary to telephone hard-Core nonrespondents. 

. Processing Survey Returns. As returns are checked in as described above, they should be place'd in the order they are 
received: district questionnaires, school questionnaires for the first school, teacher questionnaires for the first teacher 
followed by her pupil questionnaires. Questionnaiks should then be checked in sequence to makd sure that each is 
properly identified. This will be particularly necessary for the pupil questionnaires if the procedure is followed that 
calls, for the teacher (Or schol office) to transfer the teaChfir code number onto each pupil questionnaire for a given 
teacher. \ • 

Questionnaires will then be rerfdy for the next process, which for the district and school questionnaires will most 
probably be a manual (visual) exlit check' of certain key items on each questionnaire. Certain items may be 
predetermined as being, absolutely r^cessary for analysis. Should this be the case, it will ba desirable to acquire any 
missing information prior to fuither processing. In rnany situations, it will be preferable and easier to foUowup on such 
missing information imediately. Standard forms should be developed for ipailing requests for missing data, Qerks 
should make sure each of these letters receives an identification number prior to mailing. It would be preferable to 
follow sucb a mail procedure prior to introducing any necessity for expensive telephone followups. 

Questionnaires should be -subject to data captures processing as soorf as possible and on a flow basis to avoid any 
log-jams later on. Once data has been put on tape (through reliable, proven, and error-free techiiques) they will be 
ready for computerized edit checks. Previous history of the ESS points out the desirability and necessity of applying ' 
edits as soon as retains are received so that quick-resolution- foUowup can be made. Use of optical scanning on the large 
volumes of teacher and pupil questionnaires will make this possible and feasible. The first groups of questionnaires 
shoifld be run through the already-prepared edit routines to make sure they are providing the desired results. Any 
changfis and refinements should be made prior to running checks on large numbers of questionnaires. For those edit 
checks that requir^spondent foUowupfor resolution, procedures should be implemented for the periodic production 
of edit checks, folloWp outputs; and mailings. This whole process can be automated. The mailings should be sent 
directly to the districts. Telephone calls may possibly be necessary at the very end to meet project deadlines. 

Qose following of these procedures should result in high response rates on the SSES and the prompt production of 
clean, edited tapes that can be used to provide univariate statistics prior to weighting and nonresponsc adjustments. 



40 

45 



Chapter 5 



ANALYSIS OF QUESTIONNAIRE ALTERNATIVES * 

INTRODUCTION . 

* ' . • • ' 

During the design of the SSES, many questions arose as to the optimal method or approach to employ. When it was 
fo^nd that some of these coald not be resolved a priori, it was decided that t^ey would have to be evaluated in the 
field. As.a result, one goal ofthe pretest was to design and evaluate various survey alternatives. The alternatives fell into 
two major areas: sampling and questionnaire variations. This chapter will review only those alternatives that fall into 
the questionnaire design area. Chapter 6 will present an analysis of sampling alternatives in full detail. 

There were four major questionnaire alternatives tested: 

(1) response alternatives of yes/no versus mark all that apply, 

(2) response alternatives of numbers versus perceht, . . . • 

(3) selection of optimal respondent, and 

(4) collection'of welfare versus low-ipcome data. 

Two forms of each questionnaire were developed to address these alternatives. To eliminate confusion between the two 
forms, each district received the^^same form of questionnaire fpr.each response level:, district, school, teacher, and pupil. 
So. in other words, half of the districts were mailed one form of the questionnaires and half the other. 

The approach initially intended for the Analysis of survey alternatives is discussed in detail in a^parate report to- 
NCES. However, constraints placed on the field ffcst by the .delay in OMficlearance resulted in the revision of that 
approach as shoWin table 2. Table 2 presents a summary of the questioniWe alternatives tested, reviewing for each 
one the reason the alternative was ins(?rted into the pretest, indicating the analysis approach to evaluating which Of the 
alternatives should be adopted in the full-scale study, and briefly describing recommendations made by the RMC staff. 
The approach taken in this chapter will be to disCuss ,each of the alternatives in turn and to provide some background 
information regarding the basis for the recommendations madd by the RMC staff. It should be 'noted that the 
recommendations presented here^re a summary of the recommendations for the individual questionnaire items. 
Specific information on each itein^is contained on the item analysis sheets on file at NCES. 

ALTERNATIVE 1 : YES/NO VERSUS M ARK ALL-TH AT-APPLY 

During the early design stages ofthe SSES pretest, two questions were raised that led to the insertion of both yes/no 
and mark-all-that-apply response alternatives for a series of items. The first question arose because the RMC staff was 
uncertan as to which of the two'approaches would yield the most accurate data. One of the main concerns of the staff 
was that the us'e of the mark-all-that-apply format allowed no internal edit check on the data provided by respondents. 
However, it was felt that the yes/no format would permit the analyst to review each of the questions and to determine 
whether the respondent did not provide the data requested. The mark-all-that-apply format on the other hand provided 
no way for checking this since the respondents need only mark those that were specifically applicable. The second 
question leading to the insertion of this alternative c^ealt with the issue of respondent burden. It was felt at the outset 
that* requesting the respondent to check either yes or no for each of the possible choices would add considerably to the 
response burden. Since both of these questions operated in different directions--one in favor of the yes/no responses 
and the other favoring the mark-all-that-apply response approach-it was decided to insert a series of questionnaire 
Items In bath formats: half of the questionnaires to be mailed would request one form, the other half would request the 
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Table 2.-Suniniary of- questionnaire alternatives tested 



Alternatives tested 



Question addressed 



Response alternatives 
of yes/no versus mark-all- 
that-apply 

Response alternatives of 
numbers versus percents 



Selection of optinwl ^ 
respondent 



Question existed as t6 
which approach yielded ^- 
the more accurate data 

Question arose as to whether 
respondents could provide 
numerical responses 6r 
whether it was only possible ' 
to estimate the values re- 
quested and provide percents 

Question arose about 
which respondent could 
more easily provide the 
most accurate data to ^ 
specific questions 



Analysis approach 



^Analysis of frequency 
distribution of survey re- 
sponses 

Analysis of frequency 
distribution of survey re- 
sponses . 



Review of frequency 
distribution and vali- 
dation study outputs- 



Recommendations 



Use mark-all-that-apply ap- 
proach 



Impossible lo generalize^ 
must judge on individual 
basis, but general tendency 
toward numerical responses 



See item analysis shee^ 
(on file at NCES) for 
each item 



Collection of welfare 
versus low-income data 



It wqs unknown at the outset 
as to whether respondents 
would-be better able to pro- . 
vide information on welfare 
or low-income families 



Discussions with re- 
spondents during vali- 
dation study 



Request low-income data 



other. Table 3 indicates the questions that were treated in this manner. All 40 questions were entered in both the 
yes/no and mark-all-that-apply response formats-but on separate forms of tho questionnaires. An example using 
questipn 1 5 of the, pretest district questionnaire is shown below: 



EXAMPLES OF THE YES/NO VERSUS MARK-ALL-tHAT-APPLY 
^ ^ ALTERNATIVES FOR THE SAME QUESTION 

YesiNo Attermttve , 

15. Were any of the following methods or standards utied as part of the last needs assessment conducted by this LEA? 
« (Mark "Yes" or "No" fof each). a . ' 

a. Conference with parents 

b. Conference will) pupils i 

c. Staff or teacher oonfcrcnco i 

d. Survey of parents ' ^ * 

e. Survey of pupils 

f. Survey of staff or teachers ' 

g. .Pupil achievement scores i 

h. Pupil grades ' _ 

i. Pupil IQ scores . ' P 

•j. Other (Specify): . 




Mark-AfUHvMfiply Alternatives *^ 

15. ^ch of the following methods or standards were used as p3h of the last needs assessment conducted by tills 

LES? (Mark all that apply) . 

□'Conference with parents Q Survey of staff or teachers 

□ Conference with, pupils Q Pupil achievement scores 

O Staff or teacher conference ^ U Pupil grades 

a Survey of parents Q Pupil tQ scores 

Survey of pupUs u . Q Other (Specify): ' . 
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Table 3.-0uestions, tested for yes/no versus mark-all-that-apply responses 
(Question numbers refer to the pretest instruments) 



Question number . Subject of Question. 



District Questionnaire ^ 

14 Programs involved in needs assessment 

15 ' Methods used in needs assessment 

16 , • Major needs identified in nee^s assessment- 
.30 Major functions of title I parent advisory council 
SO Target groups for title HI project 
53 * Krect recipients of title III services 

56 Members of title III advisory council 

57 Major functions of title III advisory council>f^— 
72 Title VII services provided 
76 Major emphases ~of training for title Vll staff 
78 Major functions of title VII parent advisory council 

102 Services for handicapped pupils 



School questionnaire 

1 • Grade levels in school 

15 Federal projects in school 

16 Target groups receiving federally funded services 
18 Grade levels served,^ by title I 

21 Services provided by title I 

24 Grade levels served by title I migrant 

27 Services provided by title I migrant 

• 32 Grade levels served by title III 

35 Target groups receiving title III services 

36 * Services provided by title III 
41 Grades served by title VII 

44 Title VII language of instruction 

56 Orthopedic barriers in schbol 



Teachej questionnaire 

12 Primary focus of preservice training 

15 Primary focus of inservice training 

16 Target group focus of inservice training 

17 Federai programs in which teiacher pailicipates 
46 ' Type of special training to teach handicapped pupils 



Pupil questionnaire 

9 Areas of instruction 

10 Qassification of pupil 

27 Problem subject areas 

29 Persistent problems requiring assistance 

33 Services received through title I 

36 Services received through title III 

39 ' Basis of selection for title VII 

41 PupiPs handicapping condition(s) 

48 Services for handicapped pupil 

49 Equipment or materials for handicapped pupil 
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The approach taken to evaluate which of the response formats would be better for inclusion in the full-scale 
SSES was basically an analysis of the frequency distributions and accuracy of survey responses. These frequency 
distributions almost invariably yielded fewer responses for the mark-all-that-apply format. Because of the type of 
validation instruments developed, it was impossible to directly use validation data as part of the analysis of this 
alternative. ' . 

As mentioned before, the yes/no approach was included to provide an, internal edit check. Respondents, however, 
only checked yes'' responses and' did hot check "no'* responses, treating the questions much the same as 
mark-all-that-apply questions. Again-using district question 15 as an example, the responses received as a'result of 
the pretest are as follows: ' * 



• Mark-ail-that- 
apply .format 


Yes/No format 


Number of 
. responses 


• 

Percent of 
responses 


Number of 

"Yes" 
responses 


Percent of 
. "Yes" 
responses 


Cases where • 
"Yes'* responses 
exceed mark- 
all-that-apply 


3 


12 


Average: 
6 1/3 


23 


X 


10 


40 


13 


48 


k 


5 


20 . 


10 


37 


X 


11 


44 


19 


70 


X 


9 


36 


14 


52 ■ 


X 


8 


32 


12 


44 


X 


20 


80 


19 


70 


X 


17 


,68 


22 


81 


X 


7 - 


j28 


12 


44 


X 


3 


12 


7 


26 


X 


2 


8 

'■ 1— ^ 


3 


11 


X 



Response indicated 



' Omits 

a. Conference with parents . 

b. Conference with Rupils 

> c. Staff or teacher conference 

d. Survey of parents ^ 

c. Survey of pupils 

f. Survey of staff or teachers 

**g. Pupil achievement scores \ . . 

h. Pupil grades 

i. Pupil IQ scores , 

j. dther (specify): 



While it is impossible to make statements about the validity of any given item, the fact that, in 9 out of 10 cases, 
there were more '*ycs" responses than mark-all-that-apply indicated a trend on the part of respondents to 
over-report ^ '*yes" responses, a trend substantiated by validators' comments. In addition, averaging across all 
respondents to each type of response format, only 12 percent omitted responding to the que^bn on the 
mark-all-that-apply format, while an average of 6-1/3 respondents did not respond to the question on tl/e yes/no- 
format. When individuals did respond, they entered an average of 4.18 and 6.34 responses for the mark-all-that- 
apply and yes/no formats, respectively. ' ^^^S**i-* 

Reviewing these facts, and combining them witli the knowledge of school systertis obtained through validation 
study site visits, analysts determined that the mark-all-that-apply responses were more accurate since respondents 
were over-reporting on the yes/no format. Hence, it is recommended that those questions pretested in this manner 
should be inserted in the full-scale study in the mark-all-that-apply format. The revised questionnaires reflect this 
recommendation. 



ALTERNATIVE 2: NUMBERS VERSUS PERCENT . ' 

The 3SES is a very different type of purvey from that usually conducted by NCES. Many of the data items 
Included on this survey are subjective in nature and request ^ta concerning personal estimates of individuals 
involved. Tllis being the case, the question arose as to whether the respondents could reasonably be expected to 
provide numerical responses for certain data items, or whether it was only possible for them to estimate the values 
requested and to provide percentage answers. Analytically, it was preferred that respondents proyidc numerical answers, 
since base data would then be available for a number of analysis procedures. On the other hand, it was felt thaty 
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requesting' numerical answers indicated a level of precision in the data that was not necessary for the analysis to be per- 
formed and posed an unnecessarily high level of burden on the respondent. Therefore, as table 4 indicates, 12 
questions were designated to^ppear in both formats so the question could be resolved. An example of each alternative 
is shown below using question 4 on the pretest district questionnaire: 

EXAMPLES OF THE NUMBERS VERSUS 
PERCENT ALTERNATIVES FOR THE SAME QUESTION 

Number alternative . 

4. Approximate how many elemerftary pupils in this LEA are from families whose primary 
^ supporter receives welfare? (If none, write •*0") 



Percent alternative 



. Pupils 



4. Approximately what plercent of the elementary pupils in this LEA are from families 
whose primary supporter receives welfare? (If none, write •*0") 



.Percent of pupils 



Table 4. -Questions, tested for number versus percent' responses 
(Question numbers refer to the pretest instruments) 



Question number 


Subject of question 


District questionnaire 


4 


Pupils from families on welfare ' 


6 


Pupils from non-English-speaking homes 


8 . 


Educationally deprived pupils 


Schooljquestionnaire 


7 


Pupils from low-income families ' 


9- 


Pupils from families on welfare- 


10 


•Pupils from non-English-speaking homes / - . 


11 


Pupils from non-English-speaking familie^ earning $3,000 or less 


14 


Educationally deprived pupils 


•46 


Lanugages spoken by title VII participants 


Teacher questionnaire 


24 


Pupils from non-English-speaking homes 


27 


Pupih with general ability to work at grade level 


28 


Pupils who will be prepared to work at grade level next year 



This question was analyzed through art evaluation of the frequency distributions of the survey responses. These 
-frequency distributions yielded very similar data for each of the items regardless of the format (numbers or 
percents) uSed. As may be seen in the table below, however, the omi t rate f or numerical responses is consistently ; 
lower than for percentage 'responses, indicating that respondents had tewcr problems in answering numerical 
questions. ' , . 

45 . 



v^uesuon 


Omif rate (percent) 


Percentage 


Nil i mi^Hpiil 




re SO on ^e<: 




District 4 


16 , 


• 0 


District 6 . . . . . 


12 


4 


Districts 


16 


0 


School 7 \ . 




0 


School 9 


i 


3 


,^ School 10 


J 


3 


School 11 . . 


»j 


n • 
u 


School 14 > . 


9 ' 


3 


lA 


■ 74 


'77 


-teacher 24 




2 


Teacher 27 . 


"T 


3 


tfeacher28 




2 



The accuracy and validity of the percents versus 
numbers varied by each specific item tested. In ad- 
dition, difficulty was expressed in the ►field-' validation 
of 5onw*-items. However, it was determined that re- 
spondents were better able to provide numerical 
answers for most of the data items in question. As to 
the recommendation for this alternative, it is im- 
possible to generalize. Each question or item must be 
evaluated on an individual basis as shown in the item 
analysis, but there is a general preference among RMC 
analysts for numerical responses to facilitate analysis in 
terms of the aggregation of data, the development of 
averages, and in weighting individual responses. 



ALTERNATIVE 3: SELECTION OF OPTIMAL RESPONDENT - , 

The SSES is.a largc-scalccffort that, in the full-scale implementation, will survey approximately 800 school distncts, 
3,000 schools, 12,000 teachers, and 48,000 pupfls. This being the casp/it is possible to aggregate many'^data items to 
national totals and hence averages. Since the data can be aggregated frojn any level, many of the questions (such as 
enrollment, program participation, and expenditures) could be asked at whatever level was most appropriate from the 
viewpoint of both respondent burden and data accuracy. The pretest, therefore, was designed to determine which 
respondent or questionnaire was the best location for a number of data elements. In addition to the respondent'burden 
and data accuracy considerations, some items needed to be entered on more than one questionnaire to permit gertain 
types of analyses. Questions asked of multiple respondents for whatever reason occurred 43 times as shovyn on table 5. 
Shown below are examples of similar questions asked at differcntlevels of response: 

t EXAMPLES OF MULTIPLE RESPONDENT QUESTIONS 

* (Question numbers refer to the pretest instruments) 

School questionnaire 

47. What arc the major criteria used in the selection of elementary pupils for participation in the ESEA 
title VII project in this school? (Mark all that apply in each vertical column) 

' Pupils whose primary Pupils whose primary 

or dominant language or dominant language 

is English is other than English 

a. Basis of selection is unknown a □ 

b. No special criteria employed o [ a 

c. Pupirs English-speaking ability o a 

d. Scores from standardized achievement test written in English a ! ! . ! ! o 

0. Scores from standardized achievement tost written in the -primary or ' 

dominant language of the pupil n a 

f . Pupil grades o o 

g. Teacher recommendations a o 

h. Economic need of family a a 

1 . Other (Spectfy); .* .* [ '.'.[[ '. '. '. '/.'. . '. o 

Pupil questionnaire 

39. On what basis was this pupil selected to participate in tfT^HsBJtH^e VII pS|«L (Mark all that aDDlyV 
O Basis of selection is unknown ^ j^^x i-k / / 




o No special criteria employed 
o Pupirs English-speaking ability 
o Scores from standardized achievement test written in English 

O Scores from standardized achievement tost in the primary or dominant language of the pupil 

o Pupil grades 

O Teacher recommendations 

O Economic need of family 

O Other (Specify): - f^J 
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Table S.-Qiiestions asked of multiple respondents 
(Question numbers refer to the pretdst instruments) 



Duplicated items 



— ' 

District 


, . 1 
Sdiooi 1 


Teacher 


Pupil 


2 (Une a). 


2 (iino b) 




— 


.2 (lino c) 


2 (lino a) 


21 


— 


3 


6 


— 


— 


4 


9. Ua 


— 


23 


5 


4 (coi. b) 


23 


12 


6 


10, iia 


24 


i7 


7 


13 


^— 




8 


14d 


~— 


30 


9 


15 


17 


~~ 


— 


4 (coi. a) 


4 


~" 


— 


1 


-20 


I 




7,8, 11^ , 




21, 22 


— • 


12 




19 


— 


16 


— ' 


lOb 


— 


— 


27 


6 (coi. 


— ' 


— * 


28 


6 (coi. 


— 


— 


29 


8 


24 


20 


32 


— 


— 


21 


— 


33 


34 (if totaled) 


26 


■ — 




— 


27 


— 


330 


— 


22;i8(Migrant) 


31 


— 


40 


30 


— 


— 


48 


34 


35 




SO 


35 




lOb * 


51 


39 




. 37 


54 


37 








36 




36- 


71 


43 


37 






44 








45 








47. 




39 


76 




12,i58 




88. 89 


48,49 






90 


5i 




48 


94 


50 




41 



Subject area 



94 (if totaled) 50 (if totaled) 
97,99 (Unob) 52^ 



97 
98 

99 (iiJie b) 
103 



54 (Une a) 

54 (total of 
llnc5 b & c) 



40 — 

41 — 

43 — 
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Member^ip in grades: prckindcigarten-6 
Totai membership in LEA/$cho61/class 
Population size of area LEA/School located in " 
Welfare-percent of pupils from families on 
Race of pupils-number of each 

Primary or dominant language other than English-percent of pupUs 
Sixth graders reading bciow grad^iovcirpcrccnt • ^ 
Educationally deprived-percent of pupils 
Federal programs in LEA/schooi/taught by teacher 
Race of teachers 

Grade levois in schooi/gradc iovci teacher is-roporting for/grade iovcl of pupil 
Low.lncomc families-percent of pupils; definition of low income ^ 
English-speaking ability of pupils 
Target groups 

General academic ability when pupils entered class. 

General academic ability of pupils next year/at end of year 

Likelihood of completing elementary school, high school, some college 

Number of pupUs In title 1 In LEA/school/dass 

Subject or activity area of title 1 services 

Number of pupils In title 1 Migrant In LEA/school 

Subject or activity area of title 1 Migrant icrvlcei 

Selection and mode of title 1 participation 

Number of title III projects In LEA/school 

Number of pupils in title HI in LEA/school/dass / 
Target groups of title 111 project 
Selection crtterion of pupils for title 111 

Mnior emphasis of title 111 in LEA/school . v . 

Suljject or activity area of title 111 services 

Number of pupils in title VI 1 in LEA/school/class 

Language of title Vll project in school/class ^ 

EngUsh-spcaking ablUty of title VII pupUs 

Selection criteria of pupils for title Vll 

Mnior emphasis of prosorvice or Inscrvlcc 

Tills is actually a skip, but it also toils why handicapped papUs aren't getting services 

Services provided to or needed by handicapped pupil»-lheso three questions are very 
similar but also very different. District Q.90 deals with availabiUty of services 
but the last column (not available to pupils with i\ccd) Is probabiy the same as 
school I . PupU Q.48 Is probably the same as the first column (availabie to 
every pupU with need) of district Q.90. ' * 

Number of handicapped pupUs by handicap-district Q.94 and school Q^O are probably 
the same unless a pupU could be in a school and not receiving services from any 
funding source 

Total number of handicapped pupils In LEA/school/class 

Number of tc*achcrs In LEA/school teaching handicapped pupils 

Number of teachers of handicapped In regular classrooms 

Number of teachers fully certified to teacli handicapped pupils 

Number of special teachers of handicapped 
■ Number of teachers receiving inscrvlcc for handi capped during this school year 



a. Asks for primary or dominant language not Engllsli plus earning 
$3,000 or less per year-could bo arrived at through analysis of 
entries at all respondent levois on those lines. 

b. While not speeiflcally asked for hero, during analysis, this probably 
will be linked Into Federal program participation of pupU. 

c. Includes title 1 Migrant unless only operates in the summer. 

d. Mlsnumbcrcd in the cunent school questlomlaires as a Ascond 
question 13. 

c. Even though In Uic general section, the answer here would be the 
language of the title VII project. 



f. Although in the general section and already linked to school 
questton 12, It will probably also be used In conjunction with 
school question 45 during analysis. 

g. The "blllngual/bicultural" choice on teacher questions 12 and 15 
links into district question 76. 

h. It appears that school question 52 would be the same as the total 
of district questions 97 and 99 (line b). 

I it a posit hre response here, then had Inscrvlcc for the handi- 
capped during this school year. 
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The evaluation of this altejnative was more time consuming than for any other of the alternatives tested. It involved 
an intensive review of the frequency distributions as well as an indepth evaluation of the validity study outputs. The 
primary concern of analysts in this evaluation process was to .obtain the most accurate data possible with the 
secondary consideration being that overall respondent burden should not be increased: 

Using the question concerning the number of pupils from homes where the primary or dominant language is 
other than English as an example, it is possible to determine how thq contractor arrived at its recommendations. 

It was the contractor's recommendation that this question be deleted from the district and school questionnaires 
and retained on the teacher and pupil questionnaires. This was baseci on a review of the data contained in the table 
below, as well as on validator 'comments and suggestions. The' question numbers shown /m the ta|)le refer to the 
pretext instruments. 











Questionnaire 


Error 


Response 


Level of 


number 


rate 


burden 


precision 


District ,6 . . . 


24^0 


Hard 


Estimate 


School 10. . . 


35% 


Easy 


Estimate 


Teacher 24 . , . 


32% 


Easy 


Estimate 


Puj)il 17... 


11% 


Hard 


Bstimate 



In most cases, RMC analysts were able to make recommendations as to the suggested location for each of the data 
elements. % 



ALTERNATIVE 4: WELFARE VERSUS LOW-BMCOiyfE DATA 

The SSES has been designed to collect information on school systems in general and Federal program' 
participation in particular. These Federal programs are designed to reach pupils with special needs. 

One target group with special needs is generally felt to be children \vhD have been determined to be. 
educationally deprived and who live in low-income areas. Hence, it is a concern of this survey to collect accurate, 
yvalid information on the extent of low^income pupils within school districts. During the survey design stages, it was 
/unknown whether "the respondents could better provide information on the number of pupils who come from 
( families currently receiving welfare, or whether data were more readily available on the existence of low-income 
families within the district. Given this, it was decided to pretest questions on this topic during the 'spring of 1974. 
Table 6 indicates the seven questions relating to this alternative, while examples from the pretest school 
questionnaire are also shown. 



EXAMPLES OF LOW-INCOME 
VERSUS 

WELFARE QUESTION ALTERNATIVES 



Low-income alternative 

7. Approximat^cly how many of the pupils in this school are from low-income families? 
(If none, write •'0") 



Pupils 



8. What is your definition of low-incofhe? 



Welfare alternative 



9. Approximately how rnany of the pupils in this school are from families whose primary 
supporter receives welfare? (If none, write *'0") 



.Pupils 



ERLC 



48 

53 



Table 6.-Questions regarding low-income or welfare status 





District questionnaire ' 


4 


- pupils from families receiving welfare 




School questionnaire ^ 


7 


" pupils from low-income families 


8 


- definition of low-income 


9 


- pupils from families receiving welfare 




Pupil questionnaire 


21 


- pupils from low-income families 


22 


definition of low-inqome 


23 


" pupils from families receiving welfare 



The approach that the contractor's staff mombers took in the evaluation of tfiis question was to conduct 
discussions with respondents during the validation study site visits. It was learned through these discussions that, 
while respondents have difficulty providing information on ^ther topic they have an «ven greater amount of 
difficulty providing information on welfare. Whereas some school districts had conducted an ESEA title I urvey in 
the past year requesting information on the existence of low-incon^amilies within the district, in almost no case 
did the school Tlistrict officials have information concerning the welf^ status of pupils' families. 

The difficulties encountered by respondents in providing welfare data can be seen in the omit rate, shown below, 
which is double that of the equivalent low-income question. 





Omit rate 




Ljow-incomc question 


Welfare question 




Number Percent 


" Number Percent 


School questionnaire .... 
Pupil questionnaire 


1 i 

. 15 1 


• 3 2 
27 2 



m addition.'the error rate for the welfare questions was high relative to that for the low-income questions as may 
be seen in the table below. Question numbers there refer to the pretest instruments. -- 



Questionnaire number 



Error rate 



Low-income 



Welfare 



District 4 . Not asked 57% 

School 7/9 30% 58% 

Pupil 21/23 0%. 0^ 



Where requested, welfare information was either estimated or respondents were forced to contact either welfare 
or social service agencies within the school district area/ It is for these reasons that the contractor recommends that 
the questions concerning this issue request information on low-income families receiving welfare. 

54 ^ 
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Chapter 6 , 
ANALYSIS OF SAMPLING ALTERNATIVES 



INTRODUCTION^ 



•On| of the most significant sets of alternatives to be .pretested* during the mailout survey were the alternatives 
develope4 fot sampling procedures. Each, ofi^e Elementary School Surveys (ESS) in the past had followed identical 
procedures for the sampling of classrooms aiM ofWpils but these were not always successful. As a icsult, a good deal of 
attention was given to developing procedures that would solve some of th^ proT)lems of the past. 

The procedures us6d from the original purvey of Compensatory Education in 1968 to the 1971 ESS for sahipling " 
grad.es and classrooms had been to take every homeroom class for grades 2, 4, and 6 in every school in the Sample. One 
problem :this procedure offered was the difficulty inobtaining aecurat^ counts of class sections for each school prior to 
the survey, Ih addition,nhis procedure did not provide any meansof estimating between^grades variance. The procedure 
for sampling pupils traditionally: used in the ESS was to have the teachers of the selected classrooms.pick a random 
sample of four pupils in each class. The teacher manual provided instructions and a random number table/or use in the 
selection of the pupil sample. ^ ' ■ « ' * 

Given this background, RMC and NCES set out to develop alternative procedures for sampling classrqoms and pupils 
that would be tested during^ the SSES preteSt. A considerable number of alternatives werr^nsidered during a series of 
meetings throughout the surjfuner of .1973. Among the alternative procedures initlallfcunder consideration, but 
eventually ruled out, were: . . 



. • sampling by birth date across the elementary prade population of each school, 

• sampling by alpha segments within sampled classrooms, 

• sampling alternate sets of grades in ialternate schools, . * - v 

y • randomly sampling pupils from every grade in each school, and --^^ '\ * 

• sampling pupils from Jiomerooms sampled with pn^bability proportional to size across the school population. 

A basic sampling issue at one point in ilie. discussionNvas whether a one-s>age. or/ two-stage sampling design was 
appropriate. In general; the parameters followed in designing ai^ choosing sampling dternatives were; 

• The sampling procedure must provide'a means of measuring the variance between grades. Frbm past surveys, the 
variances between schools, between classes witiiin grade, and witHin classes were knowii, but it would be deisirable 

• to continue measuring these variances. " ^ 

• The total sample size should be large enough for measuring the components of sampling error, but not so large 
that the sampling error is much smaller than it needs be. NCESIndicated that the size of the \ 971 ESS sample was 
adequate to provide these measures.'' * 

• Procedures to oversample (where necessary) pppils in target pupil subpopulations and pupils in smfdl Federal 
programs should be developed. Certain programs^and pupil grotips in the past were inadequately represented for - 

^ the purpose of drawing national estimates. ^ ' . 

• Each alternative procedure developed must be auditnblcin postsurvey checks. * * 

• Each alternative must be designed to provide the necessary componejits of variance that would allow for testing'' 

' each alternative. * ^ . . • * , ' ' 

• ■ * • ■ . % 

* . ■» • 
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Th6*product of all this was the development of two alternativeicach fottlassroom and pupil sampling procedures. 
The. first classroom sampling proce^urd would selecf four homerooms at random acrd'ss all homerooms for the 

elementary grades in each school. The second procedure woiild select two grades at raiwip'm. in each school. 'In the 
. pretest, pnly^two homerooms in each selected grade were to be chosen. This wais done to cut down^on the pretest 

burden. NCES indicated that, should this procedure work,- all homerooms" in the two gr^ades selected for each school 
• would be surveyed as part of thiejPuU-scale design. , . 

-The alternative* developed,for pupil sampling was to vary the number of pupils selected in each sampled classroom. 

The alternative procedures were simple enough: In roughly half the schools, the teachers would be instructed to select 

foiir pupils. In the others, teachers would select two. If the selection of four pupils worked as well as selecting two, the 

general option was that within-class sampling would continue to select four pupils from each class to ensure adequate*" 

nun^bers for measuring within-class variance. 

.RESULTS OF SAMPLE CHECKS AND REqOMMENDATIONS ^ , 

There were three types of sarripling procedures being tested during the field effort conducted this spring. They were* 
(i) sampling of classroogis that was conducted by the principal, (2) sampling of pupils that was conducted by teachers, 
and (3) supplement-ary sampling of pupils in special target populations in some schools. These special target populations 
were migrants," bilinguals, Indians, and handicapped— and this sampling was done by the school principal; For the 
sampling of classrooms, two alternative procedures were being pretested. Sampling Type 1 (referred td as the HLF type, 
because it used^the Homerdom List Form) sarnpled four homerooms from -a list of aU the homerooms in a school. 
Sampling Type 2 (referred to as the GLF type, because it used the Grade List Form) first sampled two grades, ?ind then 
two homerooms within each of these grades. The pupil sampling procedure 'involved two alternatives, also. The first 
alternative asked the* teaoher to ^ampl6 four pupils from her homeroom;in the second she sampled only two. Th^e was 
.also a supple^nentary sampling procedure being pretested that involved sampling an adtlitibnal, number of students in 
schools that had high concentrations of these special target groups. This supplementary sample inclu^ied up to 'a 
maximum of 20 additional pupils. ' - * 

In the following discussion af the' results of the pretested sampling procedures the phrases "correct sample," 
"incorrect sample," and "procedural error" will be used quite frequently. It is important for the reader to know the 
'exact meaning of these phrases. The research design to check the selected sample 'directed the field^staff to-replicate the 
sample selection procedure that the respondent^ had been asked to complete. This replication was independent of-^the 
respondent's to ensure that the sample checker would hot be biased by the respondent's errors. After the^j^ple 
checker completed his sample selection, he compared it with the respondent's sample. If the samples were i'dentffSl, the 
respondent drew a cor^^ct sample. If they were different, the respondent drew an incorrect sample.. If the same was 
incorrect, it was. due to a procedural error, or a mistake in following the sampling instructions precisely. It ve'Ss also 
possible that the respondent drew a corf ect. sample, yet made procedural errors: Some mistakes made in following the 
instruction§ would not necessarily .affect the sample selected. An example of this would be the incorrect completion of 
the .worksheets or the omission of information requested on these sheets, neither of which led to an incorrect sample. 
Although the primary consideration in the sample' check? was that the correct sample was selected, the secondary 
consideraticn was whether or not the respondents were able to follow the instructions. ^ 

The RMC Research Corporation and OE field staff conducting the SSES sample checks visited the schools in May 
.jShortly after the respondents completed the pretest. Prior to beginning the field work, a three-day training session was 
conducted al RMC. In each school^ the sample checker independently completed the same sampli selection that was 
asked of the respondent, using the appropriate rosters and source documents in each school. He then compared his 
results jvith those of the respondent, indicated' whether an incorrect sample was selected, and, if so, the types of 
pfQcbdural errors made. Also, for each district the sample checkers completed a site visit report that summarized their 
txperienees, including problems incurred ai}d recommendations. The field staff provided many comments and 
recommendations fS redesign' of the sampling 4)rocedure for the full-scale survey in addition to their sample 
■verificatipri tasks. The sample^checkers spent approximately 2 days in each district and verified the sample selection 
'procedure of three schools ift the district. 

The package^ used by sample checkers in th^e field included the, necessary materials to complete the independent 



sample selection, materials to report correct or incorrect sampling, procedural errors, and problems found, as well as a 
site visit report outline. Also included was a list of procedural error Codes and an explanation Of each. These errors 
included those of omission, arithmetic, transcription, misunderstanding of instructions, misuse of random number 
tables, creajing lists, definitions, and other. Table 7 is a summary of the errors reported by the sample checkers and the 
frequency of occurrence* of errors by sampling type. The procedural, error types'listed above are those the sample 
checkers used ip the field and those reported in table 7. 

Table 7.-Jleported procedural errors ^ ' 



Sampling type 



Procedural 
errors 
by site 



o 
o 

a 



c 
o 

2: 



Procedural errors 



c 
o 



o 

(4-H 

o 

M 

, o 



o 

t: 

O 



e 
■s 



C 

o 

c/1 



-a 
o 

'O 



C 
3 



1 



o 

.S c 

o t3 

u3 .S 



definitional 



'o 
C 

C/3 

C 

o 
o 

5 



cr 
-a 

5 



•a • 



0 



Teacher 
packet 

of distri- 
bution 
record 



■ 3 

o 



o 
c 

5 



Qassroom 



type 1. (liLF) 


21 


• 4 


'0 


P. 


2 


13 


.1 


9 


0 


5 


0 


0 


1 


1^ 


0 


Type 2 (OLF) 


10 


IQ 


0 


2 


0 


6 


3 


t) 


0 


0 


1 

0 

J? 


0 


2 


lb 


0 


PupU 
































Type 2 (2 pupils) 


29 


37 


0 


0 


2 


0 


0 


6 


0 


0 


0. 


0 


0 


_16 


.1 


Type 4 (4 pupils) 


44 




1 


J ^ 


0 


13 


8 


17 


0 


0 


0 


0 


0 


17 


0 


Supplementary 


11 


4 


0 


0 


0 


4 


0 


1 


0 


0 


0 


P- 


0 


6 


0 



a. Sampling forms were maile4 to the district offices and had not arrived yet. 

b. ^inor procedural error-lack of thorcfughness. 



After reviewing results of the sampling procedure, however, it was possible to define a list of procedural error types 
made in following thfe HLF and GLF sampling procedures that was more concise than the list usfed by the sample 
checkers. These error types are the following: ' * • " 

- (1) Errors in creatihg'the list * -^^ 

a. Ordering within grades ' . ' 

'15. Placement of AM and PM kindergarten classes 

c. Placement of multigraded prmultiaged (ungraded) classes y 

d. Other (such as using the enrollment from a dajte other than January 31, 1074) 

(2) Transcription Errors (such as those made from transcribing enrollment figures from source'^ documents to 
sampling worksheets) ^ . 
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(3) Definitional errors 



\ a. Definitional inadequacy 

b. Definition not used * ' 

c. Definitional inconsistency - ' ' • . 

(4) ».iack of' thoroughness"* 

(5) Instructional errors ^ " 

a. Instructional inadequacy , , * " 

b. Misunderstood instructions • . . 

(6) Misuse of random number tables * ' 

(7) Arithmetic errors ' " ' ^ . i * < 

* (8) Did not fpllow instructions at all • - ' . 

(9) Other (such as not completing the sampling procedures at all) " — / 

* . 

"The following paragraphs address each of the t|iree major sampling types indepepcJently and discuss the results from 
the pretest, including the types of procedural, errors that occurred and the solutions that* may remedy them. In the 
sampling procejiures the m^jor questions that were addressed were: '^Vas the sampling done correctly?"", "Were 
respondents able to follow directions?", 'Was the procedure auditable?'', and "Which method of sampling worked 
•better?". ^ ■ / ^ ^ X 

Sampling Oassrooms ' 

As can be seen in table 7, there were significant numbers of proc^flural errp'rs and incorrect samples for 'both 
.classroom Sampling types, ^me of, these, mistakes (in following the procedures) did not affect the sample drawn. 
However, ^the number of incorrect samples was still greater than 10 percent for jf^a^/i the HLF and GLF sampling 
procedures (see table 8). ^ . • • " „ 

As a general consensus,- the sample checkers seemed to prefer the conceptual framework of the I^LF, although 
admittedly on the whole.the HLF was not nearly as effective as the GLF. In the^ 20 schools that iised the GLF sampling 
procedure, 13 (or 65 percent) drew correct samples. In the 25 schoojs that used the HLF type sampling, only four (or 
16 percent) drew correct samples. In tables 9 and 10, all thfe scHbclc t;iai were involved in the SSES pretest site ^visits 
and that conducted thq sampling procedures are listed. Designations indicate whether errors were made in^completing 
the sampling procedure and, if so> what the errors involved. . 

Type 1 -Homeroom List Form (HLF). » • ^ ' » 

-The HLF procedure involved two basic steps:' the first was creating the sample universe list of homerooms, and the 
second was selecting the homerooms from this list by use of random number tables. Most of the ejrrors made in using 
the HLF sampling- procedures occurred in creating the list. As can be seen in table 9, only two principals made errors in ' 
the second step. Four" principals did not follow the instructions at all and picked/'representative" samples based on 
their own judgment. This errorjype is therefore not attributed to either of the ^teps'in the sampling procedure. 

Of the 21 schools^that picked incorrect samples using the HLF procedures; 8 of them were caused solely by the fact, 
that they did not order -homerooms within grade correctly or place AM and PM kindergarten classes correctly when 
creating the list. No otljer procedural errors*, were made. Most respondents read the first paragraph in step 1.1 whose key 

' 58- ■ . 
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Table 8.--Classroom sampling results 





• 


1 


• . ■ > 

Sampling ^ 


Correct' 


;*> 

Incorrect 


type ; 


sample 


sample 


0 






f 

1 







Procedural errors 



V Creating list ' 



T3 



IS 



00. 



Q 



£? 

T3 
C 

c 
a> 

E ■ 

a> 
o 

s: 



3- 

E 

o 

c. 
a> 

E 

a> 
o 

s: 



c 
o 



Definitional 



o 
cd 
3 

cr 

T3 



T3 



o 

2 



•I 

3 

O' 

Ui 

O 



o 



Instruc- 
tional 



o 
Cd 
3 

T3 



T3 
O 



T3 

c 

3 



X) 
Cd 



X3 

E 

3 

C 

E 
o 

C 
Cd 



3 

CO 



q 

E 



o 
o 

s 



o 



o 
c 

S 



J2 



21 
7 



11 
0 



5 
0 



6 
0 



2 
0 



0 
0 



4 • 3 
4 0 



0 

2 



4 
1 



HLF Type L 4 

GLF Typf ^ 13 

r ' ' . ' ■ . ■ ' ' 

sentence was: "On the HLF prepare a list of all homerooms in your school in ascending order of grade level, from 
prekindergarten through grade 6 only,'' They did not read further to the instruction stating that* homerooms within a 
grade level should be ordered by size. These also included people who did not place AM and PM kindergarten classes 
correctly on the list. Some principals combined AM and PM classes and regarded them^as one class. The* procedure of 
•combining , the AM and fM classes is not technically correct, although the instructions »were never explicit about this, 
, point. Other comrfioVi errors included not using rosters from January 31, 1974, but from later dates, usually'in April 
and May. This error was caused by the delayed timing of the pretest. Except for the four principals who did not follow 
pro6edures at all but picked a sample that they felt would be "representative;" the procedural errors were minor ones 
that could be reduced by making the instructions clearer and easier, to follow. ' . 

Tha rtLF type sampling procedure was capable of being audited. All steps in the procedure were defined. The 
re3portdent was not given any xhoices where he could manipulate the sample and still follow the stated procedures. 
However, since the membership roster date that the principals Vvcrc asked to use was 3 months before they received the 
materials, there were some problems in obtaining the correct roster needed to "audit" the sampling procedure. 



Type 2-Grade List Form (GLF). 

There were three basic steps in completing the GLF sampling procedure. The first was to establish the enrollment of 
each grade level in the school. The second was to determina. which grades wei;e to be sampled. The third was to 
determine which classes within these grades were to be sampled. 

Thirteen principals selected correct samples, although some of them made procedural errors that did not affect the 
sample: Seven principals selected incorrect samples. In the first step of the GLF procedure, of the four principals that, 
made procedural errors, two were arithmetic and two were caused by lack of thoroughness. The two that were due to 
lack^'of thoroughness did not affect the sample drawn. In the second step, two of the three procedural errors resulted 
from misinterpreting the phrase "equal to or less than." In the third step, one of the three procedural errors was caused 
by an error in creating ihe list of homerooms within the grade sajjip^ed: The respondent did not order homerooms with 
equal enrollment alphabetically by the last name of the tea^^fer. Another procedural error resulted from the incorrect 
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Table 9. -Detailed description of HLF-type sampling procedure errors 







HLF- type sampling « 


I" 


• 






Step No. 1 




Step No. 2 




School 


• No 




Creating the list 


Selecting the homerooms 


Other 


Number 


errors 








Sub- 




Sub- 










step 


Procedural error 


step 


Procedural cxror 








No. 




No. 






1 




■ .1 


Ordered within grade incorrectly 








" 2 










Picked his own sample 


3 




.1 


Qrdered within grade incorrectly 








4 




.1 


Misunderstood defmition of multi- 
grade classes 








5 




.1 
.2 


Used wrong membership roster date 
Transcription error 








6 


• 


.1 
.1 

,1 


Ordered within ^rade incorrectly 
Misunderstood definition of multi* 
grade classes 

Tran^frintion tirror 








7 , 




.1 


PlaccdK classes incorrectly 


.0 


Instructional inadequacy 
- 


Picked his own slaimple 
• 


o 




.1 
1 

.1 
.1 


Lack of thoroughness (omission). 

PlnppH K rln<!<a*<! incnrrpcflv 
Ordered within crade incorrectlv 
Instrutitional'insdeaiiHCV 




0 




.1 
.1 


PlnreH, K rlncvi inrnrrnrtiv 

Ordered within grade incorrectly 








in 




.1 


Ordered within cradc incorrectlv 

lUwlwU TVILIIUI gl uUw lllVv/l IwVLlJr 




r 




1 1 
1 1 










I lujvcu Ilia UWll oollipic 


1 z 












Picked his own sampje 


13 


X 












It 














1 J 


X 












1^ 

lU 




.1 


Ordered within grade incorrectly 








1 7 




1 


Ordered within ?rnde incorrectlv 

vyiUwIwU WlLlllJI KlfclUw lllW/llwvLlJr 






Sampling procedures not 

cuiiipicic U 


18 










■ 


19 




.1 


Placed K classes incorrectly 








•20 




.1 


Ordered within grade incorrectly 






















22 




.3 


I arlf of thoroiiphnp'jQ 

x^dw ^ V/ 1 LI IV/l V/ U^l 11 1 wO^ 


.0 


Misused random number 
tables 








.1 


Ordered within orade incorrectlv 

vyiUwlwU WiLlllJl f^lClU^ II 1 VV/l 1 w V 1*1 T 






I r^niiicoH a rir\i 1 f rAmil'ir 














* vs supplementary 
banipiing pruccuurcb 










• 








.1 
.1 


Misunderstood definition of multi' 
Used wrong membership roster date 












1 

. 1 


MiQii nHerQtnnH Hpfitiitinn nF miilti. 

graded classes 








OA 




.1 
.1 
.1 


Placed K classes incorrectly 
Used wrong membership roster date 
Misunderstood definition of multi- 
graded classes 








25 




.1 
.1 


Ordered within grade incorrectly 
Misunderstood definition of multi- 









graded classes 
.2 Lack of thoroughness 

.3 Lack of thoroughness 
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Table lO.-Detailed description of GLF-saftipling procedure errors 







GLF-type sampling 








Step No. 1 


Step No. 2 


Step No. 3 




School 
Number 


No 
errors 


Establishing 
enrollment by grades 


, Determining grades 
to be sampkd 


Determining classes 
to be sampled 

'» 


Othpr 






Sub- 
step 
No. 


Procedural error 


Sub- 
step 
No. 


Procedural error 


Sub- 
step 
No. 


Procedural error 




1 


.5 


Arithmetic 


.2 


Lack of thorough- 


.2 


Creating bst 





2 
3 
4 

5 
6 
7 

8' 

9 

10 
11 
12 
13 
14 

.15 
16 
17 

18 

19 

20 



ness 



K .4 Arithmetic 



.5 Misunderstood 
instructions 



Completely multigraded/ 
team teaching (NA) 



Record of TP distribution 
not completed 



School has only one grade 
level/instructions failed 
Picked own sample 



.5 Misused random 
number tables 



.3 



Lack of thorough- 
ness 

Lack of thorough- 
ness 



Sampling procedures not 
completed 



.2 Misunderstood 
instructions 

.2 Misunderstood 
instructions 



use of the random number tables. Other miscellaneous errors included a school that - was set up on a multigraded 
team-teaching approach for which the instructions were not easily applicable. Only one principal whd was assigned a 
GLF sampling procedure neglected the instructions' altogether and picked his own ^'representative" sample. The fact 
that procedural errors made in the beginning of the process did not necessarily affect the sample drawn gave the GLF 
sampling procedure an advantage Over the HLF sampling procedure. Small errors made in determining the enrollments 
of the grades normally did not affect (he sample drawn. 

The GLF sampling proceciure had more steps and was somewhat more cumbersome in the sense that the respondent 
had t'Ospick two grades based on the enrollments in them. Then, for each grade, a list of homerooms had to be created. 
And finally, using a random selection. process, two homerooms had to be picked from each grade. Nonetheless, the GLF 
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sampling procedure seemed to produce better results than the HLF in terms of x:orrect samples. This was due to the fact 
that errors made in the earlier steps of the procedure did not necessarily affect the sample drawn. 

The GLF- sampling procedure Was capable of being audited except for one minor point. There was no date specified 
for the student membership figures used to construct the cumulative grade totals. The addition of this date to the 
* instructions would make the procedure completely auditable. ^ 

Recommendations. \ - 

2 From the results and experiences gained during the pretest of these two sampling procedures, certain things become 
apparent. First, it is imperative that the sampling instructions be made as simple as possible. Only if they are simple is 
the respondent likely to follow^ the procedure in a **rigorous'' manner. However, it is difficult to produce both an 
auditable and simple sampling procedure. Also, it is clear that most of the incorrect samples were caused by procedural 
errors in creating the list from which the homerooms were to be selected. The respondents did not have significant 
trouble in using randoni number tables or following the selection instructions. Most errors occurred ill the areas of 
producing Ijsts as of the given date (January 31, 1974), understanding how to ap^' the instructions to different 
organizational structures, using the definitions, and following the instructions. For instance, most principals did not 
consider then: special education classes as ungraded or multiaged classes, even though these classes were not designated 
as a particular grade. Thus, in some cases, the principal omitted the special education classrooms entirely from the 
sample universe list. The instructions also did not adequately address the team-teaching 'concept or the AM and PM 
kindergarten class case. 

Neither the HLF nor the GLF- sampling ^procedure can .be recommended in its present form. Generally, this 
recommendation is based on the fact that both are overly Complex. There are too many places in the instructions where 
the principal can make errors. The procedure for sampling classrooms to be used during the full-scale survey must be 
simpler, easier to follow, and involve fewer defmitions than the HLF and GLF-pretest procedures, fn particular, the 
procedure for constructing the sample universe list of classrooms must involve fewer decisions of the type as where to 
place a particular homeroom on the list or to which homeroom or grade to assign a student. It was in these areas that 
the HLF and GLF-sampUng procedures presented the most difficulty. 

' Two improved sampling procedures are suggested: (1) Sampling Xv/o grades from each school (not dependent on 
enrollment in the grades) and then using all homerooms within these two grades as the sample homerooms. This wilLbe 
referred to as "grade level" sampling. (2) Having the principal list all the homerooms (alphabetically by the last name of 
the teacher) and selecting the required number of homerooms from that list (similar to the pupil sampling procedure). 
. This will be referred to as the "alphabetical list" sampling procedure. Each of these sampling procedures has the 
advantage that it i^ much sirnder than the ones pretested. Additionally, the problems in creating the list of classrooms 
to be sampled would be esselWially eliminated. However, each still has its weaknesses. The entire concept of sampling 
homerooms for the SSES arose when it was felt that using grades 2, 4, and 6 (as in the past) did not provide enough 
infonaation to measure variability between grades. Thus, the concept of sampling grades within a school and having 
teachers within those grades complete the SSES instruments addresses the issue directly. But the disadvantage is that all . 
students in that school must b(^, assigned to a particular grade level (i.e., to a homeroom at,a particular praHe level) 
before the sampling procedure can be completed. This presents problems in the easels of team-teaching, mujtigraded, 
ungraded, special education, and open-classroom situations, where it is difficult to assign each child to a particular grade 
level. ' " 

X The alphabetical list sampling procedure-although the simplest, most logical way from the point of view of 
principals-does not stratify the sample by grade level. However, when a random sample is selected from this 
alphabetical list, the grade levels chosen will also be randomly selected. -It gives less control over grade distribution of 
the sample. For example, it would be possible, although highly unlikely, that all homerooms selected in a certain school 
would bfc from the same grade level. It is^also possible that^cach homeroom selectedln a sample school would be from 
different grade levels. SampUng from the alphabetical list of homeroom teachers avoids the problem of assigning a grade 
level to the homeroom?. However, the problem of assigning each pupil to a unique homeroom still exists. Some schools 
. do not use a homeroom structure. Iti these cases, the first class period on Monday morning or some other point in time 
would have to be used to define the homerooms. The alphabetical list procedure also has the advantage that the number 
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of children in a homeroom does not vary much across LEA's. Whpn combined with the fact that a fixed number of 
homerooms would be selected from each school, this means that the number of pupils in the sample is hiore effectively 
controlled. Another consideration is the logistical problem of conducting the survey. If one samples all homerooms 
within the grade levels selected, the number of instruments required cannot be determined in advance. Thus, an 
overisupply of instruments would have to be sent to ensure that a particular school had enough. This is a significant 
disadvantage. The alphabetical list sampling procediixe does not have this problem since a constant number of 
homerooms would be sampled from each school. 

Considering all the strengths and weaknesses of the alternatives, RMC feels the alphabetical list sampling procedure is 
definitely the best one. The instructions would be a two-step procedure. The first would be to create an alphabetical list* 
of homeroom teachers. In this process,. all students in the school must be assigned to a unique homeroom with a unique 
teacher. For the majority of schools, this process would be very simple since all students are normally assigned to a 
unique homeroom and unique homeroom teacher. For schools that use an open-classroom or toam-teaching approach 
to instruction, special instructions will be needed. In the second step, a random sample of homerooms' would be 
selected from this list. Random number tables would be constructed for each number from one to the maximum 
number of homerooms existing in a school. A school would use the table appropriate for the total number of homerooms 
. in the school for prekindergarten to grade 6. This sampling procedure is auditable since it will specify exactly all the 
steps the principal must complete to select the sample. There will be only one correct sample that can be chosen. And it 
can be verified to estimate how frequently incorrect samples occur. The procedure is statistically sound. It will provide 
a random sample of pupils from all grade levels very nearly equal to the proportion of children in each grade. This will 
make it possible to make estimates for each grade level, to calculate the variability within grades and between^grades, 
and, most importantly, the procedure will be simple. It does not involve creating special lists that depend on elaborate 
definitions of which children can be placed where. All of these attributes combine to give a very strong sampling 
procedure. i' 

Pupil Sampling * " 

For each homeroom selected to be included in the SSES ptetest sample, the teacher was asked to follow procedure 
to select cither two or four pupils. The purpose of the pretest was to determine whether selection of twcJ^r four pupils 
by the teachers would have any effect on the randomness of the sample or the probability of the teacher following the 
sarnpling instructions precisely. The sampling procedure for teachers to select two pupils will be referred to as pupil 
sampling Type 2 and the other as pupil sampling Type 4. 

Generally, it was found that problems resulting from the pupil sampling procedures were minor for both alternatives. 
First, the creation of the list from which the teacher would select the sample of pupils was straightforward 
(alphabetized)^ The likelihood tiiai a teacher would already have a list of this type was very high. In some cases, 
teachers had an alphabetical listing of their children but was separated by boys and girls. This did introduce an error to 
the sample chosen in some cases. Second, the use of the random number tables was quite easy for the pupil sampling 
procedures since the teacher needed only to know the total number of pupils in her classroom. Tliat number referred 
her to the table having the correct sequential numbers of uic students to be included in*the random sample. In table 7, 
it can be seen that the majority of teachers did not choose correct samples. However, the procedural errors made did 
not introduce serious errors in the sample picked since they did not affect the randomness of the sample. For example, 
errors in alphabetizing the list of students did not systematically include or exclude certain types of students. The 
important point in the pupil sampling procedure is that ail students must be given an equal qhance of being included in 
the sample. Thus, as long as the pupils were placed on the list and the teacher UseS the random number table, the 
sample selected was random. 

Seldom did teachers pick their own "representative" sample of s'tudeflits. Before the SSES pretest was conducted, it 
was felt that sampling of only two pupils in a plassroom might introduce problems of this type: Teachers miglit be mpre 
tempted to select p^piL that they felt were representative if such a small number of pupils were to be sampled. 
However, during the pretest, teachers commented to sample checkers^that they knew the sampling was supposed to be a 
random process. Even when they questioned the appropriateness of the pupils they had selected randomly, they did not 
let this sway them from attempting to follow the correct procedures. Still, the problem with minor errors such as 
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misalphabetizing is that the procedure cannot be audited unless there is stjfTct adherence to the instructions. One cannot 
determine whether the sample was purposely manipulated or whether a true random sample was selected unless all steps 
in the procedure are determined and followed exactly. 

Table 7 (shown previously) is a display of the results as reported by the field staff for the sample checks. It can be 
seen from this table that most procedural errors fell under the category of "other errors" with a known cause. 
Examples of the errors that teachers made during the selection of the pupil sample are: (1) not alphabetizing the list of 
pupils correctly (usually using separate alphabetical lists of bQ^s and girls, (2) using the student enrollment roster for 
the wrong date,. (3) errors caused by lack of thoroughness (sucK as miscounting), and (4) misusing the random number 

tables. - * 

* * 

Recommendations, ^ . 

The pupil sampling procef^ure used a straightforward, alphabetical list of the pupils and, depending on the total 
number of children in the class, the teacher was directed to a specific random number table. The random number table 
contained sequential numbers for the students, on the alphabetical list to be included in the sample. The sampling 
procedure was very seri3,ijtive to the number of students in the class because, for each number of total students in the 
class, there was a- different set of random numbers. The procedure could be audited. However, it is recommended that a 
sheet for the actual alphabetical list of studeats in the class be included as one of the survey forms. These forms should 
th'en be maintained in the school's survey files since they would provide a means of-checking the teacher's sampling 
procedure. It is important that the sampling procedures be implemented correctly if a nationally representative sample 
is to be obtained. Thus, the sampling procedure must be auditable so it can be determined whether it was correctly 
implemented. - ' 

^ The conclusions that can be drawn from the pretest of the pupil sampling prQcedures are iMi both pupil sampling 
Type 2 and pupil sampling Type 4 were successful in selecting a random sample. The procedural errors that occurred 
were the type of errors that would always occur regardless of the quality of the instructions or the procedure used. As 
long as the procedure must be auditable, it will retain a certain degree of complexity that will invite errors. It should be 
pointed out that even thougli the procedural errors uncovered often affe\:ted the sample selected, they, did not affect 
the randomness of the sample in most cases. Instructions should be slightly redesigned and refined to 'emphasize the 
importance of the teacher following the.procedures precisely. 

Supplementary Sampling 

The supplementary sampling procedure pretested was designed to ensure that a sufficient number of students in 
various special target groups would 'be obtajned. This was done so sufficient data could be collected to provide 
statistically <valld results. The four target groups were: migf3nts, bilinguals, handicapped, and Indian children. Not all 
schools were asked to complete supplementary sampling procedures. Only those schools that had significant 
percentages of these target populations were asked to participate. Of the 20 schools that participated in the 
suppicmentaO' sampling procedure, 15 were examinnd during ihc sample checks. Only four of these schools completed 
the procedures without error. Table 1 1 indicates the schools that picked correct supplementary samples and for those 
schools that did not,^n explanation of the problem or procedural error that caused the incorrect supplementary sample 
is given. 

The supplemepfary sampling procedure was an involved process composed of four steps: 

(1 ) compiling the list of the target population, 

(2) selecting a sampling plan, 

(3) selecting the sample, and 

(4) coiTipleting the supplementary sampling form. 

Tlic actual samp^ng procedure instructions di;Uiot cause the mjjjoriiy of the problems. Rather, 4 of 15 schools did 
not conduct th^supplemcntary ^nmpliji^^m^ccdures at all, feeling that the regular sampling procedi/res superseded 
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Table 1 1 .-Supplementary sampling results 



School 

iiumuci 


Target 
population 
type"* 


1 


!• 


2 


I 


3 


B 


4 


, B 


S 




6 


M 


7 


^ M 


8 


M 


9 


H 


10 


H 


II 


H 


12 


H 


- 13 > 


H 


14 


H 


15 


H 



No 
errors 



Explanation of errors or problems 



Sampling procedures not completed-respondent uncooperative * 
Sampling procedures not completed-noiist of Indians exists in school 

>% 

Indadequate instructions 

Sampling procedures not completed-logfstics problem 



Substituted regular pupil sampling in handicapped classrooms-misunderstood instructions 
Subi^titut^d regular pupil sampling in handicapped classrooms-misunderstood instructions 
Target group changed from Indiana to handicapped because of small number of Indians 

(4) in school-principal not notified , 
Error in creating Hst-laft one student off list ^ 
Did not complete supplementary sampling procedures-respondents thoughPnegular sampl- 
ing superseded supplementary 
Did not complete supplementary sampling procedures-respondents thought regular sampl- 
ing superseded supplementary 
Did not complete supplementary^ sampling procedures-respondents thought regulaf sampl- 
ing superseded supplemi^'tary 



*I - Indians, B - biHngual, M ■ migranf, H ■ handicapped 

them. There were also several schools that did the opposite: they completed the supplementary sampling procedures 
but did not complete the regular sampling procedures, feeling they were superseded by the supplementary sampling 
prpcedures. Another two schools knew that they were supposed to conduct a supplementary sampling procedure, but 
they merely used the regular sampling procedure in classrooms that had the target population students of interest. 
These cases occurred with handicapped pupils. Other procedural errors that occurred were caused by problems in 
finding the data sources to compile the list of supplementary samf)le pupils. For instance, many principals did not know 
the cfiteria or the information cuncerning whether or not a child shouW be considered an Indian. Also, Federal 
definitions of who should be considered a migrant pupil are 'often different from those used by local school authorities. 
, Handicapped pupils may be considered handicap pc^Hn a certain school district but not by Stale or Federal definitions. 
It was frequently difficult to determine whether a child should be considered as bilingual or coming from a home where 
the primary language is other than English. In short, the supplementary sampling procedure had many definitional 
inadequacies. 

Recommendations. 

This procedure would be significantly improved if the instructions were refined. H6wever, there are still' many 
deficiencies in the basic framework of the supplementary sampling procedure. It tends to pidce an undue burdeaon the 
principals and on a few teachers. For example, a teacher af the educationally handicapped or a teacher of bilingual 
children might be asked to complete questionnaires M all the supplementary pupilsxhosen. It also tends to be highly 
confusing because two sampling procedures are conducted in a school instead of just one. Tlius, although the 
supplementary sampling procedure could be improved to the point wherq it would be some\yhat successful, alternatives 
to the present procedure should be considered. ' ^ 
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A very desirable alternative would be to sample, schools with increased probability if there were a concentration of 
these special target groups* in them. For instance, schools with a large dndian population would be selected with 
certainty in the SSES sample. Tliis probability of selection compensates for the small number of these children in the 
public school system at large. However, the major difficulty with this concept is obtaining the necessary data about 
concentrations of these target pupils in all schools or school districts. The approach, therefore, is to heavily oversample 
those schools in which there are high concentrations of the special target group pupils to obviate the. need for 
oversampling pupils within a School " 




Chapter 7 
VALIDATION STUDY 



/ INTRODUCTION 



The original goal for the pretest of the SSES was to prepare a reliability and validity study that would evaluate 
the ability of the instruments to collect accurate data. In a decision jointly arrived at by RMC and NCES, the 
reliability study was deleted and ajl available resources were, channeled into the validity study. Tlie reasons for this 
decision were as follows: 

(I.) Tlie funds available for the reliability and validity study were limited and it vJlis felt they should be spent in 
acquiring only the information most useful for the revision of the instruments, ^ 

(2) The data requested in the instruments were of the type for which analysts expected little response variation 
over time, but there were doubts as to whether respondents had accurate information (o report. 

(3) The sample included in the pretest was sufficiently large to reduce errors in measurement (a reliability 
question) but could not be expected to reduce systematic errors (a validity question). 

Tiie basis for the decision, in a technical sense, was that for any characteristic of interest concerning q respondent, 
there is considered to be some "true" vahie (uj). When a questionnaire is used to determine the value for a respondent, 
an answer (xj) is obtained that is not necessarily equal to the true value but rather is a random variable that has an 
average value a^ and a standard deviation aj. The difference between aj and Uj is called the bias of the measuring 
instrument. Qiecking the validity of the measurement may be thought of as estimating the bias bj = ^[-^\- Note 
that in this formulation of the response error model, the bias j-eflects the effects of systematic errors, and cannot be 
reduced by making repeated measurements o^ a respondent with the same instrument. ^ 

On the other hand, the difference Xj - aj^54^le random part of the response error, averages out to zero o^er. 
repeated independent trials. The quantit^y-€cj^alficasure of the variation of a respondent's answer in repeated trials, 
if Gj is large, we say that the meiisurement is unreliable. 

Now suppose \yc attempt to estimate the average value of U| in the population by drawing a random sample of n 
respondents. Normally we would use . 

1 n 

as the sample estimate-the average of the responses for the n respondents in our sample. In this case, the estimate 
X will lioye a bias equal to the average value of the bias bj over all the N elements of the populatfon. It will also be 
subject to random variation from two sources. One source is Hie pppulation variability Op between the values aj in 
the population, and the other is the variability Oj between (independent) repeated responses from the same 
respondent. 

The total variability of x will be proportional to ^p^+ Thus if the variability between responses from the 

same r^jspondent is not more than say one-third as Ihrge as the variability between the average values aj for different 
Tespond(?nts, the resportse variability component will increase the sampling variability by only about 5 percent, since 
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^ = ap^= l,05ap. 



provided, of course, that the sample is a relatively small fraction of the total population. If were half as great as 
ap, the increase would b'cxinly.|j^bout 10 percent. , • * 

RMC, therefore, concluded that because resources for checking were limited, as was the case in the SSES field 
test^ we should concentrate on measuring the bias (i.e., the validity) of the answers rather than devoting resources 
to replicating the process to get a measure of the response variability (reliability). > , 

Once the decision was made to focus on validity issues, three overall objectives were stated for the study: 

■ ... I 

(1) to produce quantitative measures of the type of nonsampling errors, 

(2) to determine the sources from which data were drawn in responding to the survey, and 

(3) to elicit comments and suggestions iof the full-scale implementation of the survey. 



ASSUMPTIONS AND PREMISES 



basic premise for this validity study was 



i.e., the data 




data <:puld be validated in the traditional sense; 
district system an^d could be located as part of a 



field work effort. This assumption was based on RMC's previous \Vork with other validation studies in similar areas. The 
SSES instruments were reviewed to seb if this assumption would hold true in this case., Pronr Uiis, it was determined 
that, while thfere were a number of questions for which the approach would not work, for many-Qthers it would, 
id as a result, it was felt that validation would be a fruitful method to follow. 

!ith the basic assumption given, the first major step in the development of the validity study was to prep1tfC4i^ 
plan fbi^using the informatidn that would ba collected. This plan also served to determine the instruments and, 
forms to redeveloped and the precise approach that wouJd be taken. The basic method RMC decided to use in this 
validation stuoy^as to search for: 7 

• the distributibnof errors by questionnaire data element, and ^ 

"~ •^e distTibtttioTruT^e(rors by type of error. 

Therefore, it was decided to^ncentrate the Vaj^ation analysis olh those errors occurring with a high frequency. In 
the first case, RMC would reviCNMhe data oJgments in which errors were made most frequently, regardlws..aOhe 
cause or type of error. From this rc^iev^:J^C would be abje to determine which d3ta element, item, orJo^tfbn 
contained the greatest error on the SSES instruments and thus deserved the most careful review.* A review of the 
frequency of each type of error (arithmetic, transcription, etc.) wouW allow RMC to detenpine whether the need 
existed for special instructions or rpvisions. WK67ievefr Uiere were differences betNyeen the actual data as determined 
by RMC and the data reported by the school district, a thorough review could then be held to determine where the 
instrument or instructioin should be revised so that the data reported \yoiild be valid. It was realized that with the 
small sample of sch6ol districts visited, results of statistical analyses might not necessarily be conclusive. However, it 
was decided that this approach could provide informttion con* iTining the analysis of the SSES data and Wuulu assist 
in providing directions for the full-sca|c implementation of the survey. One of the majqr efforts would be the 
evaluation of comments made both by ilie respondents to the survey and jon-site interviewers. It was believed that^ 
the evaluation of these comments coifld significantly improve the full-scale implementation of the survey and thaj/ 
they would prove most useful for questionnaire revisions. 

Because of the limited time and funds available for the validation study, it was felt that it would be impossible 
to validate every questionnaire item at all of the sites visited. Therefore, it was necessary to select a subset ofJ 
to be validated from over 30Q itertis on the four questionnaires. The staff of RMC, Intran, and a consultant (Mrs. 
Nancy Bradley Royalf) rated each data item on the questionnaires against four basic criteria. Tlie criteria jJiat were 
used in the selection were: 



(I) Importance in Analysis 
0-not used in analysis 
I -used in analysis of sotme data ' 
24^used for analysis of large sections of data 
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' (1) Confidence in Accuracy of Data - ♦ ♦ 

« 0— few doubts about the accuracy of data . 
1— some doubts aboiit data accuracy / 

2^strong douhts . " ^ * * 

3— grave doubts V--^ * * 

^ - . ' ■ 7 ' ^' ■ ■ ^ ■ -.^'^ ' • — ■ 

(S) ^//brr Required for Validation 

0— easily validated • - ' , 
1 —possible to validate * r . , * * , ' ' 

' 2— considerable effort required for validation « ' ' . 

3— unreasonable effort required for validation * , , 

' * " ■ . J . . 

(4) Existence of Prior Validation Studies ^ " ^ - 

0~item valid - / ;S ' 

1 — similar item fouhd valid ' ^ 

2— item not validated on previous stytdies - ' ' * 

3— item validated on previous study with poor results * ^ 

■ ■ ^ • x\ , ■< 

The scores given to the questionnaire items by these evaluators were summed. Then candidates for validation were 

selected *by paying the greatest amount of attention ^to the importance each, item would play in the analysis of the 

SSES data. The result of ttus, effort was the determination of a subset of data items tovbe validated at all sites, 

another subset' would ,be /alidated at approxim^Tely half the sites, and the remaining data elements would be 

validated at only two sites/ ^ . ■ ^ . ^ • ' ■ , ' 

The frequency with' wW were validated' in v the sites visited may be seen in table 12 below. This table also 

indicates an estimate of/the amount of time thought to be required for the validation effort. This time estimate Sf/Sf& 

founjd to be quite accu/ate during the site visits. . , ' 

Table 12.-Frequency of item validation and est*imated time require^ 




Questionnaire 


Frequency 


of item validation 


Estimated time required 


All sites ^ 


Half 


Two 


District 


43 a, ^ 


^ 32 , 


55 ' 


2 days _ ^ 


Sqhool 


30. 


.13 


18 


4 hours 




11 


12 


38 


2 hoiUFS 


Pupil . 


, 5- 


21 


' . 34 


'l hour 
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basic approach taken in the validation of the SSES- documents was a series of site VMs to a sample O'f the 
school districts that completed the instruments in the spring of 1974. Since the site visits were to be made so late 
in the school year, the coqperation of participants was extremely important to^the study. This being th^ cas^, .tlie 
sites visited, were selected* o.n the basis of providing a broad range 6f^ school district characteristics afi^ were 
composed of those who agreed to participate in the v^idation study. The result this is^that the data that came 
out of this study may or m'ay not be statistically significant, but they -do provide broad indications of the direction 
and magnitude of the problems of the survey. / ^ . ' * 

The field" work effort ms composed of two /major parts: Fjfet, validators, spoke with the, individual who 
cojnpleted the SSES documents in the school districts to orient themselves^tq the methodology used by the district" 
in completing the forms. Through this discussion, validators learhed thing^ such as the degree of automation in the 
school district recordkeeping system, the utilit^^of State or Federal handbooks in responding to the questionnaires, 
and any significant problems encountered with the survey. Th^ second part of the approach was an attempt to- 
'complete the SSES documents from th^ basic data sources available at the, School disJrict. This allowed the* 



vaUdatorsno cornpare the data £h*ey developed with that^ent^re^d previously, by the school district respondents. So 
^ that this procedure could take place while the. validators were slill in the field, the LEA's participating in the^ 

validation study had "been asl«fcd to hold their completed questionnaires until after the site visits were finished. 
In pqxforming this,,study- RMC developecf a comprehensive package of data-coilectiga instilments. The rationale 

for this was twofold/ First: to ensure the collection of accurate data across school districts and, second, to ensure 
^consistency among.th^ field staff that would l?e used in the data-collection effort. ^e materials developed consistedoT: 

(1) Interview Gw/de/mey. ••The interview guidelines wfere developed to be used at the State, -district, school, and. 

-teacher levels. These guidelines were designed to collect information on logistics', the precan\^ass survey, 
• background information on-the school system, and problems encountered in cogipleting the SSES instruments. 

{2) Questionnhire . Critique 'Sheets, The quesj:ionnaire critic^ue sheets were developed to be used individually [or 
the district, school, teacher, and pupil quisstiannaires, and were designed to collect information on the clarity 
a'nrf wording of-bcjth.ftie questions an4 the?»definitions used, >^ ^ 

(3) Validation Workbooks. The validation woKkbpoks were developed'' individually for the . district, *school, 
•* , teacher, kid pupil questionnaires.' They were designed to collec^Mnformation on the original source from 

which the resppndent collected the SSES daja, the problems encountered; the recommendations made -by the 
RMC staff member for corrections, the value of the^yalidated data elements, and the cause or sourc'e of^etror 
i£ the element was in error. ^ * * \ 

(4) Special Pwgrqms^ Form. Special progranis fonijs were developed for the district 'Shd school levels to 
determine how Well the ^district and school " personnel were using the District and School Lists of Special ^ 
Programs. ' ' ! • • 

' ' ' - ""^ ♦ 

" Because of the time constraints imposed on the project by thS ^delayed 0MB clearance, a short period of time 
was available in which to do the wouk. Consequently, a large st^ff was required*for th,e validation study.- Included 
amo/ig ihe staff were individuals who either vvere not familiar y^ith the SSES goals and objectives or who had nqyer 
participated in aWaliSation study. Therefore, a 3-day training session was conducted by the contractor. 

As a result of the Approach taken, a number of outputs were expected, from individual validators including: / 

. " '■■ - ^ . - " . " ' . - 

(1) A completed workbook for each of the questionnaires validated in the field. 

(2) A critique sheet for eatch questionnaire validated in the 'field. ' ' 

(3) One special programs validation sheet? for each district and each school visited in the field. \ ■ ' 

(4) Interview guidelines fof each of the major persons interviev/ed. * 

(5) A corn plete site* visit Report, documenting the jjersons interviewed and the outputs of the study. The site visit 
report was to consist of a description of the school system, a desqripjion of the method by whiclj the school, 

t district completed thie SSES instruifidats; a description of tlie yalidatiog of the SSES instruments,* ah 
enumeration of the probleAis*discdvered, ^as well as the recommendations for their correction, and an 
indication of^ necessity, if any, fpr developing State-specific instructions for the SSES instruments. 

A separate* report to NCES presents a summary of the validation site visit reports *as well as the site visit repor.^ 
themselves. Listed as part of the'summary^are the sites that were visited and the major* persons contacted there. \\^ 
gives a concise overview of the procedui'fes conducted by the vahdato^s and perhaps, -more importantly, thb$euse*d / 
by 'the LEA*s as they completed the survey. Significant problems encountered by the validators and .respondents 

•alike /are also discussed. These, in coilibination with the rfecgmmendations suggested 'by the field staff and school 
district personne^^. provide valuable insights th^t shojiild go a long way' toward improving not ooly the^SES but Mso 

« other Federal data^coUection efforts. * . ^ 

* ^ . \ - • ^' 

EVALUATION OF ERRORS ' ' . ^ ^ ^ 

Where validators .found discrepancies between the dat^ they prpduced J^hemselves^nd that previously entered on 
the questionnaires, an interview was held with the original respondent to determine* the reak)nnfor the differences. If 
the differences could not be resolved at that .time, ihe^errqrs were recorded in the validation workboolcs. * « . * 
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, Considerable tho'ught was^iven to the distinction bjetween a discrepiancy and an error. A discrepandy was defined 
as a difference between the data estimate initially determined by the fndependent validator and that originally 
submitted by the respondent. If, during the resolution interview, it , was determined that the respondents' data were 
truly in enor, the discrepancy was then called an error. If, however, it was determined that the respondent was 
correct and the valicfator was incorrect, the existence of a discrepancy was not^d by the validator to provide an 
estimate of the difficulty involved in obtaining accurate data' for a given questionnaire item. 

In an'effort to standardize the analysis and reporting of 4he SSES validation study data, a series of error Ixodes 
wer^ developed. While' the categories of errors were not mutually exclusive, it was useful to categorize them for 
analytical purposes into the following types: • ' ♦ « ' * 

(1) Arithmetic Errors 

• (2)* Definitional Errors * ' •• ^ ^ 

a. Definitional inconsistencies ^ , ' • . » 

b. Definitional inadequacy . " ■ . • 

c. Definition not used ' , ' ' ^ , 

(3) Estimation Errors . ' < ^ ' 

(4) Formatting; Errors ' ^ ' . r ^ 

(5) Timing Errors . - ^ ^ 

(6) Transcription Errors ^ V ' • ^ * 

(7) Lack of Thoroughness » 

(8) Lack of Source Data . - ^ 

(9) Misunderstood Instructions 
(10) Repeat (Carry-OVer^ Errors-this includes total line errors iJiiless* they were arithmetic / 

(1 i) Other-identified as to cause . ■ • 

(12) Unknown . ^ > 

These error co'des Were defined far use in the field and for analysis purposes as follows: 

^ (1) Arithmetic Errors v/ere errors in fhe basic addition or subtraction employed in responding to the SSES 
^ „ ^ questionnaires or where a peccenftagc bf an LEA figure wasMncorrectly. calculated and entered onto the 
SSES form. . 

(2) Definitional Errors: * / *. 

a. Definitional inconsistencies occurred when the SEA or LEA used a different definition from that used 
on this survey for the same term. Examples of this may be seeri ifi the varying definitions ior preservice 
training or the various handicapping conditions. . ' 

b. Definitional inadequacy was used when the definitions provided in the SS^S instruments' appjarently did 
^ noffneet the needs for this survey. * ^ 

c. Definition not used was Qsed if the respondent did not read the defim'tions provided. " *" 

(3) Estimation Errors occurred when* the respondent had no data at hand for a given data element, was forced' 
*to*make an estimate, and used an incorrect estimating technique. . ' * 

(A) Formatting Errors were those , errors that arose, because data were maintained at., the SEA or LEA in a^ 
'different format from that required for completton'of the SS^S instrument* These types of errors were 
generally found in conjunction with one or mor^ of the other errors described. 
(5) Timing Errors occurred when more Accurate data woifld have become /available after the SSES instrument' 

was completed. . ^ * * ' 

C6) TranschptidrjE^rrors occurred when numbers were incorrectly transcribed from* one form to another. 
(7) lack of Thorqughness was used when there appeared to be no reason for an error othei^ than carelessness 
on the part of the respondent. . . 

' (8). Lack of Source Data occurred when required data were not available to the respondent. 
("9) Misunderstood Instructions was used when the cespoQ^ent did not understand the instructions given for the 
SSES instrument. , ' ' 

(10) Repeat (Carry-Over) Errors occurred when the error was solely the "result ofa previously reported error and 
generally appeared in a tdtal line ' . 
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(11) Other Errors were all errors for which a cause was known other than those above^. * * 

(12) Unknown Errors were those errors for which tha analyst was unable to assign a cause. 

The remainder of thi^ section addresses the errors that«>the field work validation staff discovered. These errors 
were uncovered as a result of analyzing the differences between data the validators foui\d while in the field and 
those submitted by the respondent. The causes of the errors were numerous; however, they were classified as be§t as 
possible within the error code framework previously defined. Di many instances; it was very difficult to cj^assify 
errors by type, but^t did provide a useful means for finding trends or errors that a large number of respondents* 
made. These errors fiould possibly be reduced by redesigning the question itern or the approach in the cluster pf 
questions related to a specific policy issue. It should be pointed out that both random and systematic errors exist in 
«the data provided by the respondents. Owing to the limited tlnrie» that could be spent at each site during the 
Validation effort, tljtesev two types of errors could not be accurately separated or "individually quantified. The 
analysis, therefore, suggested by the validation effort wa^ much more concerned with systematic errors than random 
errors. This >Vas because systematic errors produce a larger bias in the mean value of the variable than random errors 
(which, by definition, are Random about the mean and i^et to zero) ir\ general, and systematic errors usually have a 
single causal factor that could probably be corrected. On the other hand, random errdrs^have many causal factors 
that produce th« random effect. IJnfortunately, these can never be completely eradicated. ThQ analysis here will 
begin by looking at errors by error code, pointing out which errors occurred most frequently, and why. Next, 
analysis clusters will be examined. Finally, the four questionnaires (district, .school, tpaqher, and pupil) will be 
discussed, including the types of errors most prevalent in each. This analysis should provide insight into the basic 
problems with^the questiomiaires .and what migh t, be done to eliminate these pr oblems. 

"Analysis of Errors by Ei'ror Code * 

Table 13 presents in summary form— for the district, school, teacher, and pupil questionnaires— the errors found 
for those items that were validated. Figures 8 through 12 graphically display the distribution of errors for all 
questionnaires, and^then errOrs on the district, school, teacher, and pupil questionnaires, 'respectively. The error that 
occurred most frequently was caused by the respondent misunderstanding the* instructions. Jhis error comprised 
^l7.86 percent of all^Jhe error?. Some of the questions in the SSES instruments required more effort than- could be 
. reasonably expected of q respondent. This was particularly, true in the handicapped sections on the district and school 
questionnaires. However, the judgment as to whether a respondent misunderstood the instructions was not an easy one 
to mafce and some of the errorj io this group may belong to those caused by lack 0^ thoroughness. The causes of the 
latter'can be found in factors^such as using, the incorrect reference data for providing th6 data, or fnisinterpreting the 
intent of the question or key words in the question. Any complexity in the instructions for the questionnaires resulted, 
from attempts , by the SSES instrument des^pers to get comparable data across LEA's. However, these considerations 
ha^to be weighed against respondent burden andnhe likelihood of the directions being followed explicitly. As one can 
see from the tables, errors caused by lack of thcyoughness occurred 15.29 percent of tlje tim<i, which was^the fourth 
most frequent error. , ' ^ 

The second ntosi frequently occurring error was caused by definitional problems. This error code was subdivided 
into^^hjee different categories: definitional inconsistencies, definitional inadequacies, and definitions not used. The 
overall occurrence of "definitional errors was 16.76 percent of the time. 

The majority of these errors were related to definitional inadequacies. The educational system and organization 
in the United States is highly varied. The methods of teaching and approaches to instdietion are numerous. Thus, to 
design a questionnaire that wilf collect comparable data from all of these systems is very difficult. To do this, strict 
attention ^must bcgiven ^o the definition of -terms used in the questionnaires. Many definitions were placed on the 
front- inside cover of the instruments, explaining the meaning of such terms as "handicapped children," 
"educatiot)ally» deprived pupils," "ins^vice training," and **ungraded classes.^' However, differences between the 
LEA's definitions and the terms used ip t'hq pretest questionnaires presented serious problems. First, the respondents 
Tound it im'possible to us^some of the definitions provided in the SSES instruments because they did not apply to 
a given situation. The problenxof definitional inadequacy was further compounded by the lack of data sources. For 
instance, . there wa's generally no source available to answer questions dealing with (educationally deprived 
pupils-even when respondents read, understood, and used the definition for the term. As a result^ they tended to' 



Table 1 3. -l^rror summary by type of error 




a. " See the descriptions of the error codes on pages 67 and 68 . 

b. Error types 2a, 2b, ^nd 2c are subcategories of error type 2 and have been reported here to permit an analysis of the types.of defi- 
nitional errors encountered. This being the case, the total of each column wili always exceed 100% by,the sum of 2a + 2b +2c. 

use alternative data sources such as the number of children in the ESEA title I programs, or the number of children 
from low-income families that do not accurately reflect the meaning of educationally deprived. Thus, when a 
respondent went to an alternative or related data source to provide the information, he also had to manipulate the 
definition of the key words in the question. Another example can be seen in the definition of handicapped pupils as 
"any pupils who have been classified as handicapped according to State guidelines." Many LEA's and schools are 
not familiar with the State guidelines for classifying handicapped children and, in some States, guidelines do not 
«xist. Thus, the definition of handicapped pupils was, in many cases, not useful to the respondent in providing the 
needed data. More consideration must be given to the definition of the terms in the SSES instruments. These 
definitions must either be made more comparable with those used by the LEA's and schools, or they must become 
highly specific to leave no doubt about the intent of the question. . 

The third most frequently occurring error was caused by lack of source data. Many of the LEA's and schools djd 
not keep records that cbntained the information the SSES instruments sought. This could be for several reasons: the 
Federal Government or State agencies may 'not have required the LEA's to keep this type of infbrmation in the past, 
this type of information is not useful to. the LEA, or it cannot be gained easily by the LEA; The lack of source 
data for many of the items on the questionnaires was a major problem and resulted jn 15.42 percent of all errors. 
Thus, the data u'sers for many of the questions should evaluate whether the LEA's had data sources and, if they 
didn't, whether estimates will suffice. In those cases where it will not, special efforts should be made to encourage 
LEA's to collect this type of information. As a result, strong justification for these data will be necesSary. Much of 
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Figure 8.--Distribution of all errprs by error type-all questionn^res 




1 2 2a 2b 2c 3 ^4 5 6" 7 ^8 9 10 11 12 \ 



Error type^ 



a. Sec the descriptions of the error codes on pages 67 and 68. 



Figure S'.-uistribution of errors on the district questionnaire, by error type 
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Figure lO.-Distribution of errors on the school questionnaire, by error type 
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Figure 1 l.-mtribuiion of errors on the teacher questfonnaire, by error type 
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Figure 1 2.-Distribution of errors on the pupil questionnaire , by error type 
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the data concerning, for instance, the income of thfe pupil *s family, will never be easily obtained since it is quite 
sensitive information. In addition, s(5me of the information requested on the SSES instruments is of a personal 
nature, such as how long the student has lived in the United States or the age;of the teacher, etc. If this type of 
information is not already part of the LEA's information process, .it will be difficult to convince the LEA's to 
collect it unless strong justification for its use can be obtained. As mentioned in the paragraph above that discussed - 
errors caused by lack of thoroughness, response burden should be reduced if at all possible. One way of doing this 
would be to eliminate questions for which no data source is available since they provide a high response 
burden-especially if the respondent tries to generate the data from scratch. , *» 

To sum up, lack of data sources for providing information requested on the SSES instruments was constantly a 
problem, not only for the respondent, but also for the validation staff attempting to assess the accuracy of the data 
being provided. It is important to take this type of problem very seriously, to be aware that auditable data cannot 
be provided, and to consider it in any analyses or statements based on the data. For those items of this type that 
arc indispensable, it is important to initiate action that will encourage LEA'S to collect the necessary information. 

The four types of errors discussed above- were by far the most frequent. The other types of errors occurred much 
less frequently and did hot provide problems as serious as those discussed there. A fair number of errors fell into 
the category of unknown causes because the validation of many items was very difficult if not impossilJie. In some 
cases, when the viator compared the results of his efforts with those provided by the respondent, the differences 
between the responses could not be resolved, but no clear reason for this could be established. A fair number of 
miscellaneous errors occurred where there were known causes, but they could not be coded *into the framework 
provided. These areMhe types of errors that will always occur in any survey. No one type of miscellaneous error 
occurred frequently enough to cause any major concern; however, ^their sum was fairly substantial. But it does not 
appear Uhat *any systematic bias was introduced by these miscellaneous errors, a fact that reduces concern about 
them somewhat. - , 

Repeat or carryover errors comprise 5.27 percent of those errors reported.' These errors resulted from errors made 
earlier in the questionnaire or earlier in the process of providing the data. These errors occurred chiefly in 
conjunction with arithmetic figures where an arithmetic error was carried over from calculation to calculation. 
Arithmetic errors occurred nearly as frequently, with 5.54 percent of all errors. Tlie burden of arithmetic , 
calculations in the SSES instruments was minimal and, although the occurrence of these errors cannot be 
completely avoided, they are not considered to be a major problem. 

Format errors, or errors caused by source data being in a form different from that required by the questionnaires, 
was rarely used By the validators, accounting for only 0.40 percent of all errors, but as can be seen from the 
preceding discussion, problems of this nature were often coded as Other types of errors. Certainly, a problem for all 
surveys is asking questions in a form that will apply to all respondents. Some respondents will have the data in a 
form close to, but ndt-exactly the same as that requested. And when it is riot the same, respondents must adjust 
their figures to reflect fhe data required on the questionnaires. However, for some questions where the data 
requested was in a form consistently different from the way the LEA's kept it (such as questions on the district 
questionnaire dealing with per pupil expenditure), revision should be considered. 

Timing errors comprised 4.19 percent of nil errors largely because the SSES instruments solicited data for the 
current schm^l^ear. Much information is not available until the final reports are written for many of the Federal 
programs-after the school year is completed. Thus, for example, accurate figures for expenditures and revenues do 
nbt become available until the end pf the fiscal year. While it appears that this is sometliing that cannot be avoided 
since cufr<Jnt data are needed, it is not a major problem for many data items in that estimates.Ju:c-sMsCfTcientIy 
accurate' to meet user need. Tlius, some accuracy will be sacrificed by having current school year data available 
quickly. / 

Another timing problem unique to the pretest was that the questionnaires were sent out substantially after the 
baseline date of January 31 used in the questionnaires. Tlierefore, many responcients provided data for later in the 
year than January 31 -usually late April or early May. Tliis problem can be avoided in the full-scale implementation 
of the survey by sending out the questionnaires in early February near the baseline date to be used. Also, 
implementing the questionnaires at this time of the year will probably reduce the number of errors caused by lack 
of thoroughness since respondents are less busy earlier in the year and they will be mbre likely to give a higher level 
of effort to providing {he data. v 

Tire only other error types not yet discussed are estimation errors, which occurred 5.14 percent of the time, and 
transcription errors, which occurred only 0.27 percent of the time. 
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Estimarion errors relate to other problems encountered in completing^e SSES instruments. For instance, lack of 
data sources or data sources existing Ln different formats from that requested are two examples of what can cause a 
respondent to make an estimate in which he becomes prone to estimation errors. Many of the questions in the. 
SSES irtstruments requested estimates or approximations. This was done to relieve the respondent of a certain 
amount of etfbrt where it was felt data might not be readily available. In other words, if the respondenty^s highly 
familiar witfi the information., his estimate, would be just as good as a "hard** data source. But the validalfors y/ho 
found data sources for some of these items jmay have also found differences' between what was prjem ded' by the 
respondent from general knowledge and what was documented, lij^ummary, estimation errors occurlfed infrequently 
and when they occurred, they could not be considered .serious Since the question ask^ for estimates or 
approximations and, therefore^ the respondent followed the instructions. 

Transcription errors occurred more rarely th^n any other type of error, wfiich^is as^pected. However, if more 
data sdilirces were avaUable for the respondents to provide the information, transcription errors might increase 
somewhat. ' ^ ^ ' • ^ - 

Analysis of Errors hy Clusters y o - 

The SSES instrujpents provide data for more than one user; thus, the questionnaire has several areas on which it 
collects data. Thesp areas are referred to as clusters of questions. In table 14, the percent . of errors for each cluster 
across all items validated i5 given for the four questionnaii4s. The clusters include descriptive characteristics; needs 
assessment; program information, including ESEA tHles, I, I Migrant, III, aod VII, and programs for the 
handiesapped; pupil achievement; and educational broj^dcasting. In this section the relative error rates in each of 
these clusters will be discussed. Some conclusions/that can be made from these analyses jjiay be helpful in 
considerifig the precision of the data and the existinice of data sources. 

^Table 14.--Frequency of errors 
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3 


11 


27 


63 
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^ote: Blank cells indicate tliat certain clusters of questions on the pretest instruments wcfb not aslced at ail levels of response. 
Frequency of validation is defined as the total number of items in each cluster that were validated multiplied by the number of times 
the items were validated. • 
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For a validator to determine that an error existed, it was necessary for him to* have access to a data source 
against which to compare the respondent's data. Thus, in those sections that contained hard information, it was 
possible to validate a much higher percentage of questions than in sections where the information was more 
subjective or source' data was lacking. In the research design, RMC did not attempt to validate very subjective or 
attitudinal-type d^ta mpre than twice. Other^ questions containing hard i data, however, were validated more 
frequently. Thus, for questions where the validator was able to obtain souirce documents and enter figures from 
these documents, not only was it possible to validate more questions but also to discover more errors. Many of 
these errors, however, were quite small. Conversely, in the sections where the information was less concrete, the 
frequency of validation was much lower and the percent of errors tended'lo be smaller since sources did not exist 
\for checking. This fact should be kept in mind when comparing the figures in table 14, Also, in keeping with the 
r^isciarch design, some questions were validated at all sites, some at one-half of the sites, and some at only two sites. ^ 
This aiCect;5 the frequency of validation seon on the table. ^ 

The descriptive characteristics section in the district, school, teacher, and pupil questionnaires contained faii^ 
hard and factttal^nfomation. Thus, this section of the questionnaire was relatively easier to validate than other 
sections. There are^^xcgptions to this statement, however. For instance, question 8 on the pretest district 
questionnaire solicited theSnujiber of educationally deprived pupils from elementary grades and was almost 
impossible to validate. Still other"qti«siions were difficult to answer and source data did not exist to substantiate or 
repudiate the data provided % the respotrdeiit. The error rate in this section was higher than in any section on the 
distric'Hxuestionnaire and was relatively high orTth^^hool, teacher, and pupil questionnaires. A relatively high error 
rate indicates that even for c^ucstiQns soliciting harShiatait is very difficult to obtain highly precise figures. For 
instanccAucstion 1 on the pretest district questionnaire ask&^for the number of public and nonpublic schools in 
the di^rict. An error rate of 42 percent was found there, >testof these errors were related to problems in 
establishing the number of nonpublic schools and the^grade split requiSsi^The descriptive characteristics sections 
of the school, teacher, and pupil questionnaires had lower error rates for>WD reasons: (J) the respondents were 
more familiar with what they were providing data about, and (2) there were not a^sqany data sources for validating 
the questions at respectively lower levels (i.e., there were fewer data sources at the scha^level th^n at the district 

and even fewer data sources at' the, teacher level than at the school, while the data sourc^Sx^ the pupil level were 

* / 

similar to that at the teacher). / 

In summary, the general cl^atactcristics section indicates that, although the data are relatively nK;|ual, they are 
not highly precise^.. To assp^s'the adequacy of the data for the user, the precision required by the u^must be 
established. The error rato'^in tliis section also suggests the difficulty in obtaining comparable data by me^im of a 
mailout survey. HowQ5^cr, it should not be misunderstood that the information provided by the respqndent in>l^cse 
sectij^^s was inadcc^rfMp^ What is' being pointed out here is that most of the figures had small errors and that the 
data are sufficiently /'depurate to meet user needs. ^ \ 

The error rates .iK'^fte various program sections-whichjnclude ESEA titles I. I Migrant, III, and VII, ^s well as 
programs for thc,h!(i1?ftlicappcd--werc relatively equaLon the school and district questionnairCjs but were smaller in ' 
almost every case ,<|^' the teacher and pupil questionnaires. This resulted from the fact that more readily available 
information warf'|s)ccd at the teacher and pupil levels. Usually the information asked for participation in subject 
areas taught, etc. It was also, very difficult tb validate the information at the pupil and teacher level sinctJ the 
number of pupils involved, etc., could, not be determined before the end of the program year when a report is 
prepared by the project director. Most of the data sources for program infomiation Were at the district level. This 
data included p/imarily prpgram applications and in some instances enrollment sheets, interim reports, or 
memoranda. At the school level, there was very little program information except for what was available from the 
principalis general knowledge. However, the principal was usually quite familiar with the programs in his school.' 

Of all the program areas under consideration, the section for the handicapped was the one that produced the 
highest error rates and the most problems. Tfiis was mainly caused by the definitions of handicapped children and 
their handicapping conclitions and the fact that school districts tend not to maintain records on handicapped pupils 
by type of handicap. Also, sources of revenue for handicapped programs arc usually mixed and not easily separated. 
Further discussion of problems encountered with the handicapped sections on all of the questionnaires can be found 
in a separate report to NCES. 

The lasi two clusters of questions were pupil achievement in the pupil questionnaire and the educational, 
broadcasting section in all questionnaires but the pupil. The error rates for these sections Were quite low. In the 
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,case of the pupil achievcmeni section, this appears to have been- caused by lack of source data for validation. Tcbl 
scores are generaUy kept in the pupils' cumulative folders and many validators were not allowed access to them. In 
the cafe of educational broadcasting, many LEA's did not have this type of program in operation. Therefore, their 
answers to the questions were either negative or skipped because they did not apply. Thus, the error rate is far 
lower there than in sections where responses had to be made. It can also be said that it was almost imp^ble to 
validate negative responses. For instance, if a principal said that there was no educational broadcasting progmiiT>in 
his school, there was no practical way to verify that theres.actually was one in the school unless it was encountereif 
accidentally. 



Analysis of Errors by Questionnaire 

In table 13 (shown previously), the distribution of errors by typg can be analyzed for each of the questionnaires. 
In figures 8 through 12 (also shoy/n previously), the distribution of errors by type is graphically represented for 
each questionnaire. There are some differences in- the distribution of errors for each questionnaire. In the following 
section these differences will be discussed. 

The most frequently occurring errors in general were those caused by respondents misunderstanding instructions. 
However, on the district questionnaire, lack of source data was the most frequent error. On the teacher 
questionnaire, definitional problems produced significantly qiore errors, while on the pupil questionnaire miscel- 
laneous errors with known causes were more common than all the other classified error types. 

The Mack of source data problem on the schopl and district questionnaires did not seem to cause as much trouble 
on the pupil questionnaire. The teacher is much more familiar with her classroom and her pupils than d district 
survey coordinator can be with all the schools and teachers in his district. Tlius, the need for source data is imch 
less critical for the teacher to provide information about herself, her classroom, and her pupils. Tlie exception may 
, be questions regarding the pupil's family. However, this reason made it more difficult to validate -responses provided 
by the teacher on the teacher and pupil questionnaires. Although the teacher was familiar with her classroom and 
her pupUs, if errors did occur, they were more difficult to detect because there Was no source data for the validator 
to use. 

The percent of errors on the teacher and pupil questionnaires was significantly less than on the school and 
district questionnaires. Again, this fact must be tempered with. the difficulties in validating data on the teacher and 
pupil questionnaires. However, the questions asked of the teacher were significant^^ less difficult than those asked 
of the principal and district survey coordinator. This was caused by the fact that the scope of information requested 
wqs much larger . for the school and district questionnaires ihqn fdr the teacher and pupil questionnaires. The 
exceiJtion to this was in the case where a pupil had transferred out of a teacher's class to another school. However, if 
the child was on the teacher's class liple as of January 31, he was eligible to be selected in the pupil saitiple. When this 
happened, the teacher sometimes needed to go to other schools to get information about the pupil. 

Balancing the low error rate on the pupil and teacher questionnaires as compared with the school and district 
questionnaires is the increased number of errors caused by lack of thoroughness on the teacher and pupil 
questionnaires. Jhis can be attribut(rd to the fact that one of the responsibilities of principals and district personnel 
is the completion of surveys. In addition, their time is not scheduled as that of the teachers for instructional duties. 
This observation points to the importance of keeping the response burden for the teacher lower than t^iat for the 
principal and district personnel. The teacher has less time to sperrd completing surveys of this type and is therefore 
much less likely to do a good job in providing, the information. ' / ' 

RECOMMENDATIONS FOR THE FULL-SCALE RELIABILITY AND VALIDITY STUDY 

There are tWo types of errors that enter into national estimates based on data collected by the SSES. Tliese 
errors are sampling errors and response errors. If the sample drawn is not a random sample, it may contain a bias 
and not be representative of the United States school population. Tliis bias may affect the estimates based on the 
data collected in the survey. However, this section will be concernjpd only with response errors or measurement 
errors in the instruments-the other type oC error with whic'h survey designers are concerned. Within the area of 
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response errors^ there are two measures -of the accuracy of the data reported on an instrument. These measures are 
reliability and validity. - ^ . * 

The reliability of an item on a ^questionnaire refers to i^s response stability. Formally, this type of reliability is 
referred to as time-associated reliability. The ideaf mettod for measuring reliability, is to obtain two measurements 
of the sam^ item by leaving an individual complete the instrument twice. This method of measuring the reliability df 
a questionnaire item has two assumptions. First, it assumes that the readministration, of the question is completely 
independent of the first time it- was administered; that is, the resfK)ndent does not have a memory carry-over. Only 
if this is true will the two . tests be independertt. The second assumption is that the value the respondent is providing 
will not have changed in the period between the administiation of the two items.^ If it has changed, Jhen the two, 
•quS^tipns are eliciting different responses anS therefore should not be "compared as two measures of the same 
quaR^iy, OtKei;%\va^of testing tl^e. reliabrlity of an item include personafinterviewing^ split halves testing (the case 
yvhere seVrnin^y.ufi^lated qu^stioHS in an instrument solicit the same. responses), and alternative forms testfng (the . 
'successive administration of t>vo pa'ralfd fbrms of Jhe same fnstrument). The latter two methods of testing the 
relfebility of an Instrument are usually only appljeabl^ tn attitu^ foTTneasurinf'the subjective values of ^n 

individual.* ^ v . • " - • - - — ' C 

It should be noted that the Teliability of an item does not express the systematic errors made in it. Thus, if tHe 
condifions that create a measurement error (which are normally chance occurrences) exist in the first and secoj^d 
adfninistration of the question to the -respondent, the error introduced Vill ;iot' be identified. The reliability <rff a 
Questionnaire, item,- howeyer, does serve, an important' function in relating the amount of variation owing to 
measurement 'versus true variation in the sample population, - ' 

The second way to measure tlje accuracy pf the data reported on a given instrument is the validity o,f th^ items. 
The validity of an item on a survey instmment refers to the question of whether or not the iji^fn obtains the 
response 'that was sought. Invalid questionoaij-e "items are usually characterized by a .systematicjsfror or bias in the 
response. -This^ of course, is different fromj'errors resulting from an unreliable question that elicits inconsistent 

.responjses. The cause of an invalid- questionnaire item' is*" usually the. fact that something in the question or 
instrument is correlated with a'Variable unrelated to what the question is requesting. , * 

An instrument or item in an instrument may be Considered valid or invalid in several ways. First, it can exhibit 

"face validity, meaniiig that it measures what it appears to be measuring. Second, it can exhibit internal validity, 
which means that checks or responses, within a question or item are logically consistent. And third, it can exhibit 
external validity, which means that the responses to a question are consistent with external sources or some external 
measure of objective re^ity. Basically, 'then, validity is concerned with whether the data obtained frdm a certain 
question are^qual to the true value of the data being solicited*. Thus, to perform 'external validity checks on 
questionnaire items, the true value of the data repbrted on the item must be known^so a comparison between the 
item response an^ reality, may be ma^. 

As discussed previously, a validitykudy was conducted on the pretest questionnaire^This validity study-served 
two purposes: (1) to/ pretest the validation' effort and (2) to gain insights into daia availability, problems 
encounterojd in^[i£ji^d', etc. While the second purpose for the pretest validation effdrt cinnot be truly^onsidered 
part of a validity study, it was undertaken tb make|recommendations about questionnaire items as a part of the 

^etest. Th§ approach taken in the validity study was to train field staff in the task of externally validating the data 

/provided by th,e respondejUs for a b^sic set of questionnaire .items on *the various instrumjbnts'. The task of the 
validator w^s to meet with the respondent ^nd 8q review the data sources used by the respor}^ent in completing the 
SSES documents.^ The validator could then compare his results with those of the respondent. If differences were 
found, he discussed the probable cause for these -differences with the respondent. The pretest validity study, 
therefore, was an atteppt to measure the external valrdity of the items submitted by respondents. Validators used 
SQurce .documents available at schftol district and school building levels ^s their measure. of objective reality. The 
approach taken was'^similar to an audit in that record-checks were the basis for the validation study./ Because of this, 
it was difficult for, soi3je items to be validated, since tlje type^of infQrmatioil elicited by them., was subjective in 
nature. |n gases where the validator could not obtain data fromAasic records for comparison purposes,, an attempt 
was made to gain as rhucb information as possible regarding the types of data sources that did exist* to substantiate 
or refute the information, provided by respondents. - ^ ' " 

As discussed earlier, RMC did not conduct a reliability analysis of the questionnaire items because the validity 

^check effort was. considered tp be more importaat than a reliability check of the responses. The basjs for this was 



the juagment that response errors would be significaRtly greater fhan sampling errors- and thus the emphasis should 
be <placed on validation as opposed to reliability studies. • ' 

Problems were incurred with ^'soff data in the validity study conducted on the pretest of the SSES instruments. 
Whereas the record-check approach to assessing the accuracy of the'data provided by respondents proved very useful 
for those- data items that were not. All school districts maintain extensive records'for .decisionmaking and reporting 
purposes as a part of their normal information system. However; many questions txji the ^SES instruments request 
_ mformation that is not of the type that has been traditionally documented. Further, much data Regarding special 
programs are not available until the completion of t"he academic year. The SSES instruments, however, were 
unplem'ented in AprU and, in future surveys, will be implemented even earUer. Thus, much of the data provided by 
respondents comes from a working knowledge of their educational system. This approach is reinforced ijy- the 
' wording on the SSES- instruijients, which .frequently requests estimates or approximations. Therefore, the approach 
of conducting record checks to assess the accuracy of the" data provided and thus 'the validity of the instrument 
posed serious problems for a number of the questionnaire items. . 

The SSES instruments that will be used in the ftiU-scale implementation of the survey are basically similar to 
those implemented during the pretest. Revisions, of course, have been made based on what was learned during the 
pretest and on changes in user requirements. The instruments include questions that require both "hard" dafa, such 
« as the number of schools or pupUs in the school system, and "soft" data, such as the number of children' from 
low-income families and judgmental or opinion data such as whether a child's home environment is educationally 
. supportive.. The instruments will again be applied at the school, district, principal, and teacher levels. At the latter 
•two levels, there is respectively ;less documentaWe data, bu't, on the other hand, there is more initimate knowledge 
of. the. respondents' componenfof the eSucational system. Since the instruments have two major bases,- questions 
regardmg pupil target groups and questions regarding Federal programs, .the instruments are divided into several 
clusters of questions that are related. TTierefore, the task of assessing the reliability and validity of the questions 
must employ a multifacetgd approach depending on 4he type of the question and the precision of the data that the 
user requires. It will be impossible to use a single approach>uch as the record-check validity concept) to assess the 
adequacy of the dgta provided on the instruments. ' For items th^t cannot" be validated using a record-check 
approach, another method must be employed. One approach that can be used is personal interviews. Ehlring 
vaUdation interviews, it can be determined \yhether the respondent understood the question and the method he used 
to provide the estimate, giving the validator the ability to -determine the level of coilfidence he has in the d3ta. 

The rtiost important question that affects the development of a reliability and validity study for the SSES 
instruments is the desired or tolerable accuracy of the data being produced. From" RMC's point of view, the answer 
to thi& question centers alpost entirely on the use to which the data wUl be put. As represented by figure 13, the 
allowable Wnge of error (or confidence) is a function of the type of decision that concerns the poUcymaker. At- one 
J extregie, an accurate, indication of program activities or estimated cpnsequences is needed for final decisions 
inVolying the, operation of specific programs. Among these decisions are those related to legislative or funding issues 
(excluded from policy-oriented concerns is program auditing where exact data statem^ are required). Other 
decisions are ,not as crjtical/ahd "ban accept (or may be forced to a^ept) ^ increasing range of error and 
uncertamty in data estimates. M the other extreme, the policymaker who must choose between alternative A and B 
as shown in figure 13, need oi^ly have data aUowing him to correctly identify which alternative program is better In' 
other words, th"e sensitivity of the measuring-instrument or the validity study (with its accompanying cost 
.implications) should be matched to the difficulty of the measurement task and the required level of data accuracy. 
Spending excessive time .and money to improve data accurScy may preclude the policymaker from being able to 
make use of the available- data. Instead,-the policymaker ' would be forced to make decisions jjased on data that is\ 
older and of lower quality, or worse, without relevant data at all. . ' - 

The key factor in the development of the reliabifity and validity study is the analysis plan for the questionnaires 
themselves. Once the analysis plan has been developed, the analysis rationale vyill be.known and it will be possible 
■ to determine which items or clusters of items, need to- be validated. In some cases, it will be necessary to establish 
, item-by-item validity: for instance, where it is necessary to know the exact number of pupils within a school system 
'or to determme the existence of Federal programs. In other cases, however, the instruments rp'ay have been designed 
to elicit generaUzed feelings or indications of activities: for instance, does the pupil in quesfion Have needs that are 
^ being met by , the ESEA title I project, or what is the teacher's attitude towards the child? Questions of this nature 
may be validated by analyzing responses to a cluster of items and relating this analysis to the objective reality as 
determined in the field. ' , • " 
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Figure 13. --Data accuracy relationships 
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TTie fact that the SSES is planned for biennial implementation and the instruments themselves tend to be sta^c 
in nature in that instrument changes tend to be evoluntionary rather than revolutionary, suggest speyial 
considerations in the- development of a reliability and validity study for. the fuU-scale survey. The reliability and 
vaUdity study over the long period of time can be an iterative process in which the approach is revised as more 
becomes known about data accuracy and requirement?. For instance, during the first implementation, the! key 
analysis questions can be identified for each cluster of questions. The data items from the questions that, are- most, 
important can be assessed for their reliability and validity. Then, in succeeding years, other items can be included in 
the analysis, while the items that have been shown to be reliable and yaUd in previous years may be just 
spot-checked. Obviously, because an item has been shown to be vaUd and reliable in 1 year, it does not necessarily 
foUow that it will be reliable and valid again 2 years later. Monitojing should be cpnducted on items that prove 
reliable and valid to, be ce^ain-Jhat no problems have arisen in the data. Basic stah|ty:is- assumed, however, unless 
there are major changes in State or Federal policies. ■ W\' , j ■ r.u 

Based on the previous discussion, there are a number of implications for the refeility and validity s^udy of the 
SSES instruments foMhe full-scale implementation. Brieny,'the major implications ai'e: y ,j . ^ 

• many items on the SSES instruments cannot be, externally validated through the use of record checks; \ 

• a Marge-scale, but not intensive', validity study needs to be supplemented with an indepth analysis of the 
Validity of instruments at a small number of LEA's; 

• a reasonably sized, but nor large-scale, reliability study is required;. and 

• it is vitally important to know the intended .uses and required precision for the data to design an effective 
reliability and valillity study. ' ' . 

It was readily apparent 'from the prete'st of the SSES instruments t^iatmuch of the data is not avaUable on- 
source documents. $ince the states of the information systems in school districts is .never static and normally 
'increases in scope, it is possible that this "situation will change -especially is the SSES is conducted bienniaUy. For 
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the present, the fact that data are not currently ai-ailable must be remembered when planning 'the reliability aiid 
validity study. For the full-scale' survey, a number of validity analyses should be performed, one of which will be 
the record-check' approach. The instruments should be reviewed carefully: and those items for which a record check 
is feasible should be identified. Then, a subsample of these questions should be selected for validation, depending on- 
their importance to the user for .analysis and on their importaijce to the user as to their precision Thus only a 
sample of. the questions on the SSES instruments need be selected for external validation through the use of record 
chqpks. . ' 

To >iuccessfully complete a validation study, the" validators must be higlily experienced and famiUar with the way 

.school distncts operate. They mustjje bowledgeable about information systems, organizational.structure reporting 
requjrements, and legislative programs about which the SSES is collecting information. Ip this Way, the validators 

X wU be capable of extracting data "from their assigned school districts that .will be 'useful to the validation effort 
Vali^ator^ sl^ould be knowledgeable about the -ppssible data sources in school districts for various pieces of 
tnformatiQp. They should be familiar with the types of personnel that exist in scffcol districts and'the types of 
information that are available at each level. Most important, they must have the .ability to- judge the quality of data 
that .can be obtained from various sources. Experience and expertise of the validation staff is essential to an indepth 
study.ot the \^hdity of the SSES instruments onsite. In the initial implementation of the full-scale SSES it is hkely 
that a large number ofi the items will not be record-check validated. Th<?se items will be the one's requiring 

^respondents to make estimates of quantifies they have a reasonable feel for, but no recorded supporting data An 
imljortant'measure^of the Adequacy -of this type bf data will betthe reliability of the question or the coaststency of 
the responses ^ven by the Respondent. Therefore, it is important- to have a reasonably sized reliability study for 
these types bf qvest.oris!^sg,Jor questi,ons. soliciting-attitudinal or opinion information, the problem of respondent 
memoty^garryovef from the 'ntst to tf,^ second implementation will not be as severe as is typically the case for 

.hard-data questions. The reason "for t|j|is that whep t>e respOndent^oes not have a hard basis from which to 
answer the question .and must make.-^^J^timate f'rom his general linowledge of the situation, he is less likely to 
remember the estimate ojiginally pro^^idfe'd.^Based on the" reliability coefficient for the items that cannot be validated 

. through a fecord-check approach, the dal|>user will have an estimate of the proportion of variability in responses 
caused. Ijy random errors versus the popqlai^ctn variabiUty or sampliiig variability. 

As to the size of the full-scale reliability 'iind vahdity study, the following recommendations are made:. ' 

, (I) The reliability study should be'co;j;ducted by mailout on a selected number of items to 100 school districts 
. ..With one school . and one teacher jparticipaling in each. This size is based on the necessity of providing a 
sample large enough to permit, valid conclusipns "yet isonof so large as to impose a significant respondent 
burdeq overall. • . .■ 

(2r The two-stage validity study . should c%isist of (1) if mailout validity instrugient, again using selected items 
to 50,^chool districts with/)ne schooJ>-and one teacher e^ach, and (2) an intensive site visit validity study at 
30 school districts. This size is required to give the iostruments a thorough analysis and may be reduced in 
subsequent year?. '• • 

•The last major implication for the reliability and validity study of the full-scale SSES is the importance of 
^knowing the intended use of the data and, thus the level oF precision required. Only with tliis information as a 
reTereftce point can a validity study be effectively conducted. The level of precision required must be explicitly 
stated, preferably in the questionnaire its(M, so that the respondent and'vaTidator alike understand what is required 
If this IS not dqne, the data reported on the instruments may be valid by some external definition and yet not 
usa.ble for the data user's purposes. • . • ' 



Chapter 8 

ANALYSIS PLANS FOR THE ' " 
FULL-SCALE SSES: RECOMMENDED PROCEDURES 

iliis ehaptcr provides NCES with a set of procedures for developing the analysis plar)S^for the Statistical Survey 
of Elementary Schools (SjSES). The chapter has two major components-first, how Analysis plans for large-scale 
surveys should be developed and, second, how NCES can develop such plans given the current status of the SSES 
development. References to specific question numbers throughout this chapter refer to the pretest questionnaires 
and are given for the purpose of example only. 

DESIGN AND USE OF ANALYSIS PLANS 

Without question, the most important component of any large survey is the analysis ^hn, for without it, the 
survey developmefrt, implementation, and resulting data will be useless. If the data analysis plan is completely and 
adequately Resigned, the writing, revising, justifying, and reporting of data* items can be done witfi accuracy and 
ease. - , - . 

There are four major steps in analysis plan development: (1) determinlation of general data and survay needs, 
related to current policy issues; (2) determination of actual data elements and development of the specifications to 
define data acceptability; (3) production of data tbble shells;* and (4) production and use of data, and data tables. 
What is important here, is that the design and implementation of survey questionnaires do not occur.bntil the first two 
steps are completed. A detailed explanation of each of these steps is provided be|ow. 

Step 1 : Determination of General Data and Survey Needs 

There are three areas of concern when defining the general scope and purpose of a large-scale survey: (1 
overriding general purpose or purposes; (2) the general policy questions to be answered; and (3) the theories behind 
the questions to be asked. ' ' ^. 

In any large-scale study, the user must first determine why the study is necessary. National survey^ are often 
conducted to evaluate the implementation 6r effects of an existing program, to monitor an existing program to 
ensure that Federal requrrements are being satisfied, to assess current unmet needs or current niethods of prograni 
implementation so that new legislation can be written or present legislation revised, to uncover whqt new methods 
or theories merit national exposure, or to provide data to the public. Whatever the perceived reasons for a study, \t> 
must be understood by and com'^municated to all respondents and oisers; all subsequent work should be consistent 
with this decision. 

Once the general use of thp study is defined, the survey users must determine the specific policy questions that 
the data collection will address. Close attention is necessary here to ensure that these policy questions are consistent 
with the study purpojjes. In instances where the general purpose of the survey is either evaluation, monitoring,' or 
needs assessment, the [jolicy questions should be tied as closely to the current Fe^deral legislation and regulations as 
possible. In fact, if the study "is an evalimtion or monitoring survey, all policy questions that do not have a basis in 
the current, legislation or proposed logislitive revisions should not be^ included in the study design— unless the study 
serves rnulti pie purposes. In research st^qies, all questions and theories to be studied should'b^ clearly defined prior 
• to ra/ther work. The -only Hiffftrflrr7^r^«Jagfwf>^n_ 2p|icy ques tions stated for legislation-based studies an^^those stated 
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for research studies is that policy questions for research studies need not be restricted to legislative issues. In any 
case, policy questions or theories siiould 6e as clearly and completely stateS as possible. For studies designed to 
provide general needs assessment or general-purpose data, the problem is not one of specifying policy questions, but 
of limiting' the scope of the study. To overcome this, a topic for the general-purpose study, such as 'teachers" or 
. "expenditures," should be selected and this decision strictly adhered to. The SSES is basically a monitoring study 
for the programs involved. \ • 

With the general purposes and policy questions for the study defined, all major policy question-related theories 
must be specified to e^isure thai the^study includes questions or optiohs to verify or refute the most important and 
relevant theories. " ' ^ - : . . _ _ . 

EXAMPLE 1: POLICY-RELATED THEORIES 

General Purpose of Survey: Monitoring ESEA title I implementation. , ^ 

T^liejrtjuestion 1: To what extent are the ESEA title I participants those children who 

have the most pronounced educational needs? 

Rationale for Policey Question \ \ Legislation requires participant children to be selected on the basis of 
V . - educational deprivation and inqome. 

USOE Policy Theory vl: ^ Most children participating in E3EA title I programs are educationally 

deprived, but there are still some instances of incorrect targeting be* 
*cauise selection is m^de solely on the basis of economic need rather 
than educational need. / > 

Rationale for USOE Theory 1 : The ESEA title I legislation has both economic and educational 

selection requirements depending on whether districts, schools, or 
pupils are being selected. There is a history of confusion of these 
^ factors resulting in selection* of pupils on the basis of economic rathfer 

than academic need. Testing the USOE -theory would reveal the 
extent to which this educatioHal/ccbnomic confusion still exists. 

After these policy-related theories have been ^ated, only those theories that, if tested, would reveal correctable 
problems or add new essential facts, should actually result in additional questions options on a data 
questionnaire. * » 

Areas of Conqern ii) Step 1. . 

At first glance it seems that NCES has perfo|TTied several of the required tasks that comprise step 1. However, 
the following concerns still exist: * \1 

(1) The general purpose of the SSES has'Shever been clearly define^d and cornfnunicated to the USOE 
participants. The survey has monitoring, ^search, and general questions; at the same time it is used for 
summarizing £EA needs assessment activi^s. Even though the SSES is a consolidated survey, it should still 
be limited to one or two purposes. ^s^^^-^jAi) ^ 

.(2) Policy questions have, not been developed' for all programs included in the SSES; all survey participants 
should be required to provide them. In developing these questions, survey participants should remember that 
policy questions need not be legislatively based if the SSES is to serve research or general data purposes^. 

(3)' No attempt has been made by data users to define policy-related ^theories to -ensure that all relevant issues 
have been considered in^the survey design.* 
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Step 2: Determination of Actual Data Elements and Development of the Specifications to Define Data 
Acceptability 

Once the policy issues and general purposes of the survey are known, the next step in di^^veloping. analysis plans is 
specifying the details of the analysis. Each 'policy question requires one or more different data elements and,, 
bptionsf; these should be specified in as much detail as possible by the appropriate data user. ^ 



y Policy jQucstion 1 : 

Data Elements Required 

\ ... 



EXAM^^LE 2: SPECIFICATION OF DATA ELEMENTS 

To what extent are the ESEA title I participants those children who have the 
most pronounced educational needs? 

: ^ Pupil participation in ESEA title I by subject area of participation. 
Pupil's educational need by subject. 
PupiPs grade level and performance by subject. 



All data elements necessary for a policy question should be included in the specification of data elements. Note that 
^the data cements specified may result ii/thc development of more than one question on the survey questionnafrcP 
or may rcsW in the inclusion of dara elements on more than one questionnaire. Once the djfta elements are 
determined, thp data user should identify the specific options and/or questions needed to provide the data elements 
and address thcXpolic^ questions tha/ are specified in the preceding steps. 



\ EXAMPLE 

\ 

Policy Question l^: 



SPECIFICATION OF^DATA OPTIONS AND QUESTIONS 

To what extent are E§EA titje I participants-those children who have the most 
pronounced cducatioilal needs? 



Data clement: ' \^ Pupil participation in ESEA title I by subject. 

Survey Question TopicV ^Pupil participa^on in academic subject in ESEA title I. 

Question O^m: ^ \lark All That Apply ^ ^ ^ 

\* Readtng ' <^ 
V Mathematics ^ 

• \9nguage 

• Oih^acgdemic ' ^^ 

. Mo^t childr^ participating in ESEA title I programs are educationally 4^rived, 
but there arcCstill some instances of incorrect t^irgeting because selecncnrTs 
matle on the basis of the economic need of the entire school rather than the 
educational need \f an individual pftpil. 

Pupil's economic status. 

Teacher's estimate of art^iual family income. 

Indicate Min/Max Income hovels 

• Under $2,000 



Policy Tlieory 1 : 



Data Element: 

Survey Question Topic: 

Question Options: 



. • $3,000 

• $4,000 . 

• $5,000 

• $6,000 

• $7,000 

• $8,000 

/• $9,000 or more 



•\- 
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The detail required by the data elements 5hould be easily provided by the data user (assuming the data user knows 
just what data, are actually needed)!^ 

Once the scope of the data needs are fully •^defined, the specifications for data accuracy may be written. There 
are «niany factors that contribute to decisions on data acceptability, and any decision that is made based on an 
accurate understanding of these factors will probably be correct, provided a conscious decision is made. Among the 
factors that should be weighed in this decision are availability of alternate data sources, measures of reliability and 
validity, use and interpretation of the dqta, and effects of sampling errors and other statistical errors of 
measurement. ' . 

In reviewing the effects of these variods facts, one must remember that, except in case^ of extraordinarily 
inaccurate data, it is possible to use almost any data With any level of acduracy if one knows how to report and use 
it within the framework of an allowable error necessary for user decisionmaking. Interpretation methods that may 
be used with potentially inaccurate data are defined in. step 4. 

The nfiost important factt)r governing the development of spepifications for data accuracy is the need for the 
data. If inforijiMion is required to address an urgent issue and no source c?cists for it, almost any data that can be 
provided for decisionmaking by a study is preferable to a complete lack of data, provided these data are reported 
cautiously. In this instance, even though acpdracy is important, availability of data is evcfi more important and some 
leniency in 'data accuracy specifications cpuld be tolerated. On the other hand, if data are available from alternate 
sources and if the limits of an issue haVe been well defined in previous studies, refinement of data accuracy and 
strong control on statistical error is of paramount importance. Specification for data acceptability should then be 
stringent. , * ' 

Reliability and validity are of major pj^Qjidkte when developing specifications for data accuracy. If a question - 
used in the same reporting situation Consistently provokes the same mean response across a subpopulatidn in 
repeated^trials, the question and. the data that result are reliable. If the response provided to any given question is 
an accurate representation of the condition being reported, the data are valid. The major problem of the SSES and 
most other ISrge-scale mailout surveys is that iEhe reliability and validity of these surveys cannot be easily 
determined either positively or negatively. ^Although the lack of reliability and validity data does not mean that the 
survey data are inaccurate, caution should always be used in interpreting data-for which the accuracy is unknown. 

The other question concerning reliability and validity is, "At what level are reliability andT validity required?" 
Many survey items tiiat are inaccurate on ah individual respondent basis may be very accurate when reported as a 
nationally representative characteristic. As discussed later in Step 4, interpretation of the data may compensate fpr 
some problems in reliability and validity, but all spedfications of data accuracy should include some mfention of 
agceptable reliability and validity measures. ^ 

EXAMPLE 4. SPECIFICATION OF ACCEPTABILITY OF 

. . ^ DATA RELIABILITY AND VALIDITY 

«- 

Teacher Question 1: What is your highest level of academic achievement? 

^, (1) A doctoral degree 

(2) A master^s degree or equivalent 

(3) A bachelor^s degree 

(4) An associate's degree 

(5) No degree but some college courses 
V (6) A high sch9ol diploma 

(7) Other 

Statement of Hypothetical Validity: 95% of the responses to teacher 1 must be valid as 

verified by school district records. 

95% of the responses to options 1-3 and opinions 4-7 each 
^ taken as one unit \yill be valid as mjiasured by reviews of 

school district records. 
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Other alternative data accuracy requirements ar« possible. As this example indicates, different validity statements 
may be written depending on the intended use of the questionnaire item. If the analysis requires distinctions 
between doctor's and master's degree holders but distinctions at the bachelor's level and below are unnecessary, the 
validity statement .would require 95 percent accuracy in responses to options 1 and 2 separately and options 3-7 
taken as one unit. In checking the Validity of options 3-7, responses made to option 3 that should have been niade 
to option 4, 5, 6, or 7 are not invalid because the analysis treats all responses to tliese options as the same response. 

In developing the specifications of data accuracy, consideration must be'given to errors caused by sample design 
and data analysis. If very stringent or rough estimates are required by the analysis, these requirements must be 
considered before sample design. A good example is asking data in terms of percents versus actual numbers. A 
respondent may be able to provide rough estimates as percents. However, the accuracy of that.specific item may be 
such that percents are appropriate for user needs and decisionmaking. Once the sample is determined, it is usually 
too late to change the sample design to meet the analysis requirements, 
f Any or all of the factors described above contribute to accuracy specification. Almost any level of accuracy is 
acceptable as long as' the extent of the inaccuracy and. its effects on the analysis are known by the data user. 
Accjiracy statements need not be made for every data item individually, but each item should be reviewed against 
* the accuracy statements that are developed in advance ^o ensure that "more stringent requirements are not necessary 
for specific item uses. Further discussion of the recommended procedures for the development of the reliability and 
" ' validity ^tudy may be found in chapter 7 of this report. 

Areas of Concern in Step 2. , ^ 

^ - - --^ 
'(1) USOE users should provide the*" -data options and elements to the contractor for each policy question and 
policy-related theory. Currently, this work is flone by both the user and the contractor in a less than flilly 
efficient manner. I • ' 

(2) Data users must specify jfequirements for data accuracy. 

(3) Reliability and validity studies that are adequate to define the extent, of SSES inaccuracies must be 
performed. ' - n 

Step 3: Production of Data Table Shells 

If the data user has provided the speciHcations outlined in steps 1 and 2, the contractor or the Federal project 
monitor should be in a good "position to direct the developnl^ht of the first draft of the questionnaires without 
undue strainl During step 3, questwnnaire development and table-shell production will run concurrently. 

Since steps 1 and 2 are now completed, the analysis plan is fairly well d^ned and is developed to the^oint that 
' questionnaires can be designed and the data user can specify the actual table shells required. Considerations in this 
phase include data intervals to be used in analysis, methods for expressing percentages, what to do with missing 
responses, item combinations or indices, and data accuracy. 

For questions 26^ 27, 31, and l33 from the pupil questionnaire used in the pretest, the examples below indicate 
, the different types of specific analysis tables that can be developed to lest the policy questions and^olicy theories 
that are used in the previous exaiiiples in this report. Discussions at the end of these examples will give the needed 
^ inforrnation for understanding how the factors listed above affect table shell design.' 
^ To address the policy question regarding educational needs of ESEA title I participant children in the simplest 

^ — format, a table like that in example 5 would be used. ^ 

This table allows the user to quickly and crudely measure the needs of ESEA title I parti^ripftnts and to provide a 
t rough answer to the policy 'question. Note the percentages in the table. Percentages are being computed across the 
) * row so that the user may readily determine the percentage of all participants who have academic need and th^ 
percentage who do not have these needs. If 'the user wished to report on the percentage of needs being met by 
ESEA title I, the percentages would be computed down the column instead of across the row. Also note that in this 
table, raw 2 and row 3 do not answer thp policy question lender study, but rather provide a comparison standard 
against which ESEA title* I data may be compared. For instance, of, what use is the knowledge that 45 percent of 
the ESEA title I participants have academic needs if the user does not know that 37 percent of the nonparticipants 
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and 39 percent of the total population have academic needs? At the same time that this table is designed, the data 
user or analysis designer should designate thp exact data element for each cell of data to be provided. 

In providing the details of data item use, the user will note that all responses or combination or responses to 
pupil question 26 and to pupil question. 31 are accounted for in the^^table. This requirement should be checked very 
carefully for eVery table since programing of these tables will require consideration of and instructions for every 
possible combination or responses. 

EXAMPLES: DEVELOPING TABLE SHELLS 
TO ADDRESS POLICY QUESTION 1 



Pupil has 
educational need 



Pupil does not have 
educational need 



Total 



ESEA title I participant 



Non-participant 



Total 



# 

% 

# 

% 

# 



# 

% 

# 

% 



# 
100% 

# 
100% 

# 

100% 



Number of missing: _ 
Total in population: 



Percent of population missing: 



EXAMPLE 6: SPECIFICATION OF CLASSIFICATION- 
OF RESPONSES IN SIMPLE TABLES 



NOTE: The table below is the same as the table in Example 5, except that information con- 
cerning the sources of data for each cell have been provided. 



Pupil has 
educational need 

(pupil question 
26 = yes) 



Pupil does not have 
educational need 

(pupil question 
26 = no) 



Total 



ESEA Title I participant 
(pupil question 3 1 = yes) . 



Non participant 
(pupil question 31 



no) 



Total 



ERLC 



Number of missing: Nonresponse to either pupil question 26 OR pupil question 31 



Total in population: Sum of total and missing 



Percent of population in missing: Missing^divided by total of pupulation 




I 



In examples 5 and 6 you were shown the simplest possible analyses that would permit the user to answer the 
policy question. This simple analysis leaves one with many unanswered questions, some of which may be addressed 
in more specific policy questions or related theories and some of which should be raised and addressed in the 
analysis of this .question. In example 5 the user receives data that allow him to determine the extent to which all 
participants have at least one academic need in some unspecified subject in an attempt to provide an estimate of the 
number of correctly targeted children. However, the fiii^t cell (row 1, column 1) also includes any re^pondciU^dio 
participates in ESEA title I reading but only has an academic need in mathematics. Would the dp<a user wish to 
count this type df response as correct targeting? Conversely, cell three (row 2. column 1) whicMncludes children, 
who do TTOt--participate in ESEA title I but do have educational needs, also counts children ill noa-ESEA title I 
schools and chil'dren in ESEA title I schools who have academic needs not served by the pa^kiular program in their 
school. Would the user want to infer that these students were incorrectly excluded from ESEA title I participation? 
An example of an additional, mor^ complex data table that clarifies the analyses for the ESEA title I policy 
question is provided in example 7. , - / 

EXAMPLE 7: TABLE SHELL FOR POLICY QUESTION 1 



ESEA Title I participant in 
reading 

ESEA Title I academic program 
participant not in reading .... 

Nonparticipant 

Total 

\^ 

Number of missing: 

Total in population: 

Percent of population in missing: 



Pupil has need 
in reading 


Pupil has 
other ;ieed 


Pupil has 
no need 


Total 


# 


# 


# 


# 






% 


iod% 


# 




# 


# 


- % 


% ■ 




100% 


# 


# 


# 


#. 


% 


% 


% 

1 


100% 


# 


»# 


# 


# 


% 


% 


% 


100% 



Here, pupil needs in reading arc matched to pupil participation in reading. Similar tables can and should be 
produced using pupil partiaipation in any one or all combinations of reading, mathematics, language, and other 
academic participation matched witb pupil' need in the same subject area. 

Policy theory I, which concerns mistargeting of ESEA title 1 programs to poor pupils rather than nee^pupils, 
requires review of two dimensions-academic and economic need-since poor children with ^z^i^i^vc^f^^^^^ are 
correctly selected and poor children with no needs are incorrectly selected. There is a variety of^fxJSsible tables to 
address this, including several variations of example 8 shown below. At tlie-v^ry^least, Ui^.&anl'c table design should^, 
b<? used to produce a table in reading (shown below), mathematics, and language.. Combinations of subjects such as 
participation in reading only, mathematics only, language only, reading and matheratftics, reading antl language, 
mathematics and language, and reading, mathematics, and language could be comp^d with the same combiftation • 
of needs. Detailed tables using combination items such as those outlined ab^ provide useful insight^ for the 
sophisticated data user, but shQuld not be developed without accompanying siptpler'tables. , 

Once the tablc^ have been completed and all options and item instru^ns provided, the data analysis plan is 
essentially complete. Individual questionnaire items and the resulting dafa analysis tables should be revised dn the 
basis of pretest data, but all changes in the study should be made w^th the data needs, analysis plans, and policy 
questions-in mind. - \ 
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/ EXAMPLE 8: DATA ANALYSIS TABLE SHELLS FOR POLICY THEORY 1 

Pupil has reading need Pupil has no reading need Total 



(pupil question 27, 
option a - yes) 



(pupil question 27a is 
or pu^pil-qa^stion 27 is 
is legitimate blank) 



Pupil *s family income is Pupil's family incomeJs 

$3,000 $6,000 $3,000 $6;00o' 

Under to or Under to ox 

$3,000 $5,999 more $3,000 $5,95^ more 



ESEA Title I participant 
in reading (pupil 
question 33a = yes) 

Nonparticipant in ESEA 
Title T re ading 



s ■ 



"^IJjupiLguestion 

33a = nQ;-pupffqiJcsUon^^ 
IS legitirn(3t^^lank)^ . . 





Total 




Number in jpissing: Pupil question ^or pupil question 27 or ptipil qu^ation 33 is nonresponse 

Total irv p opulation ^" ' 

percent of popmefkon in missing: 




Areastjf. Concern ir^ Stop 3. 



(1) There should be'fM participatio 
and table specifi(/ation^.^Us( 





d^er, and the contractor in the development of analysis plans 
tpleresls should be identified and should then be enlisted in the 

detiiiled table de\|elopm^ent. 

(2) Although a contractor would be able^TlMpdependently develop tabjo-ijhjjlls and item analysis sp(jcificalionS, 
details about intcfvals, gombinaticm'§; crosg''^^fla%ns^ to be run, and questionnaire iterjis should be 
developed inVojijuBfction vwtli the daja.oiserr^'^^N. 
'"(3) Survey redevelopment must begone on the basis oN^andysTT^plrnt?^^^ should be. 

revieweds^against thtj analysis plan before revision. ' 




'Step 4: Produp(i6n and Use of Data and Data Tables 

De\iek^pmeij|f and ^pr ct^^^ of the survey ^uestio^maitesjs^mp^ the developnfier^Kand pretcs 

aptiVfties that rw^^ncurrently ^vith table shell produclion^F^sTC]^^^^ activitie^-in step 4 are those' 

associated with surveylm^lc^jientaticfn, editing, and data tape prodij^ion/R^UaBiltt^^ studies shtiuld also 

be completed and the results used to revise the final analysis p\^^^T\mxip\o\fi the jn^pieftretU^ phase of step 4, 
.the data analysis contractor shoyld program and check the programs forj^roduclion of the data afultysis tables. 

Upon complej/ipn of the edited data tape, the d^ta anabjfsiWables should be produced and tli^r--aiialytic 
interpretation begun. In using the data tables, the following rules should be considered: 

^ ^ . 90 ' 
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(ly Dita tables should be reviewed for accuracy in terms of labeling, data aggregation^ weighting, and correct 
illicit data immediately upon production.- In* cases where the data congregate into one or two categories, new 
/ '■ t tabfesdma^ be required before the anal/sis proceeds. 

es should be reviewed and interpreted pven if the information presented in a, specific table will 
•* not be reported. This review allows the user to gain a more complete feeling of and understan^ng for the 
^ situation being analyzed. " , 

(3) Report tables in the clearest and simplest formats ppssible. Most of the public reading a report will ndt^have 
sophisticated statistical understanding, ^nd simple formats should be used as much as possible. 

(4) Report the data accuVately. Pupil questionnaire responses do not provide data -on pupils but rather data on 
teachers' impressions at pupil characteristics. Tliis distinclibn is important because teachers make many of 
the decisions regarding ph>jjUs a(^d their learning experiences. Careful interpretation and^^porting is essential 
in all analyses. Never ovcrsta5rth*-€afi£. 

1^) ^rite reports (and to som/ extent analysis plans and table shells) to minipiize the effects of possible 

/reliability and. validity proble 
1.(6) To decrease the effejcts of data^^ali^ity or reliability problems, develop analyses that require consistent 
reposing across several siniilar items. For example, before pupils are definitely classified Ipx analysis'as being 
in need of bilingual education, classify them by* developing an index based on race, target group, ability to 
speak English, and existence of persistent problems in language. If a pupil is determined to have need in^all 
four categories, the user can be fairly cehain that the pupils in question have definite need for*1)ilingual 



EXAMPLE 9: USING DATA ACGlfltATELY WITHIN TH? BOUNDS OF 
RELIABILITY AND VALIDITY PROBLEMS 



If the user must analyze Q|lpil question 24' pn economic need (see Ex^mDleo),^he should choose 
options for the analysis to rjunimize the effects of possibly invalid data. TTj^^tions used for economic 
need ar<^: / 

/Definitely poor {family income of less than $3,000) ' 
Possibly poor^ (family inco^ie'of $3,000 to $5,9S^) 
• Probably rtot poor (family ii^dome of $6,000 or more). • ^ * ^ 

1 ; 

These categories are tabulated froi|T pupil question 24 as follows: ' * 

Definitely poor - Teacher's estiijnat^of maximum family incoijie is less than $3,000. • 
(pupil 'questipif 24, column 2, options 1 or 2 are marked) 

Possibly poor - Eithgr-leaCherb estimate of minimttm family income or of maximum family income 
both^estimates are within 4he $3,000 to $5,999 range. 

(pupil question' 24, column 1 or pupil question 24, column 2 have, at least one of 
— ^ options 3*5 mar)ced) . ' ' * 

I . ! 

Probably not poor -i Teacher's estimate' of m/mmMm family income is $6,000 or iT|ore. 
• * ' (pupil question |24, column 1, at least one of optiops 6-1 1 is marked) 

Unknown 



- TeacJWS^^tim^tcs not in above categories. 



By establishing categoilo's jn th^^ fashion, the user is analyzing the data in a manner: that measurably 
increases the likeliliood of valid dsita. CarefuMnter^etation of these data and categories will also help 
ensure valid reporting. Note that'/he orfly definite Statement made is in the "definitely poor" category 
and that criteria used to place respondents are not only very stringent but alsd give the benefit of thq 
doubt to the respondent. ' / 



services. Converselv, if the pupil has no needs as measured in all four categories, he can safely be considered . 
as definjtcly not needing assistance. All other pupils can be classified in an ^\jnknown, but- may be needy** ' 

category, ' ^ 

(7) Check questionable data against alternate outside sources such as Census data, HEW Civil Rights data, etc. 

Areas of Concern with Step 4, " * 

(1) There are few user^ who are likely to write reports on this survey except NCHS. All data requestors should 
be required to specify how the data will be used and then use Tt. y ^ 

(2) User procedures should be developed to ensure release of data collected by the SSES to internal USOE users. 

(3) Federal program repTorts should be written either by or in close coordination with Federal program managers 
to ensure accurate understanding of the data needs and program purposes of the program being reported. 

9 

PROCEDURES FOR DEVELOPING SSES ANALYSIS PLANS WITHIN CURRENT PLAN 

Now that the steps for analysis plan development have been outlined, the problem becomes one of applying this i 
procedure within the current' SSES developmental situation. Much of the work that should have been done on the 
basis of a completely developed aaph'sis plan has already been completed witJK6ut its benefit. It is still possible to 
ct)ntinue with only min«i1^ revision oi'Jhc past efforts, but the first step that/nust be taken is the development of a 
gooc) and complete analysis plan. NC^3 must lake ,3 strong uncompromisi^ role in requiring USOE data users to 
justify each data item, and NCES and USOE data users must reali/e J^at an outside contractor can develop an 
analysis rationale only aftpr user need has been specified. # ^ - 

Once the analysis rationale has been determined, a contractor ma#ake the lead m preparing the analysis plans 
or fn doing all table-shell • production, programing, and table producp)n. However, as only USOH usCrs fully know 
what is needed USOE dat^ users must provide a review function fyfthe contr>^ctor in the following areas 

' ' / ' ■ 

(1 ) Determine and^communictite the general purpose of the Survey. ' 

(2) Designate the policy questions. 

(3) Specify the policy-related theories.- ' , ' 
(4X Review the specifications for data acceptability. 

(5) Review the data elements and dala options for each piilicy (jueslion 

(6> Provide guidance on table-shell speCificatioifi that inohide designation ol intervals, cross tabulations, 

categories, and items used. 

(7) RjVu'w final ddta analysis plan against protest information. ^ 

^ ' (8) Review edit spccificuhons against data analysis plan. 

(9) Review dala tables for accuracy and acceptable data variance. 

(10) Review or provide major input or interpretation into dala reports « 

To assume its proper role and to assist appropruitely in the devchyiinent of the analysis plans," NCFS, the^ 
contractor, and USOE dala users should convene a meeting of USOE management and data staffs to determine what 
the role of each will be in the development of analysis plans. 

USOE, NCI'S, and all data user-^ musl perfiirm ^le tasks iMed below at appropriate times during the 
developrhenl ol the analysis plans. ^ 

(1) Determine the general scope of the survey. Curren'tlv. the SSES performs evaluation, monitoring, needs 
asscssm(Mil, research, and general dala collection Tliese multiple use. can be carried off if necessary, but the 
current survey has encountered some design problems because of these multiple usages. The SSES attempts 
to perform many functions a fact that adds to the survey complexity and problems. Future SSES's should 
be moie limited in scope. . . 

(2) Kecjiing the general survey purpose in mind. USOE users sfioul^ review the policy questions wjth the 
C(Tntract()T and' ensure that all (juestions arc legislatively based and ^ha^. ^.'^i^U^^ legislative tissues are covered. 



92 



9 ■ ' / 

If the survey is a research or general-purpose survey, the scofse^should be carefully defined and the Questions 

and theories to be addressed should be clearly specified. ./ 

(3) All policy-related issues that are legislatively relevant should be defined and cleaily stated by USDE users. 
These theories should have major importance, result in the possible 'exposure of correctable pfotflems, and 
require the addition of survey items that would not have been included otherwise. 1 

(4) Data elements and options in a general, unpolished format should be developed by the dat^ usefrs for each 
policy question and policy-related theory. Thesis will be used by the contractor to deveTop the^estionnaire. 
items and by the USOE staffs in the next l^tep. If current items are acceptable, USOE users should explain 
the relationship between each questionnaire item and each policy question. 

(5) USOE users should devise general specifications for data accuracy for the data elements and options specified 
in step 4. These statements need not be devised for every item, but should reflect the statistical accuracy 
requirements of the survey users. 

(6) NCES must develop rules governing the review of data tables and the release of data to internal USOE users 
and release by USOE users to the publ^Jncluding the issue of whether a feedback system shoulcl be 
'developed and, if so, what it should entail. ^ 



If all of these activities are performed adequately T)y USOE datla users, the contractor should be in a much stronger 
position and should be able to deVelop data analysis plansj^data tables and data printouts, and write reports with a 
high degree of accuracy and utility. Therefore, the contractor would also be in a much better position to provide 
NO^Sajid' USOE with the desired support in these arei^i. 
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Appendix A 
PRECANVASS REPORT 



PRECANVASS TASK ^ > 

Several information elements were taken into consideration in designing the way the sthool sample was selected. 
Most were familiar: total number of schools ir^ the LEA, school enrollmenlt, grades served, enrollment by grade and 
type of community (urban, suburban, or rural). Since^he SSE^ concerned several specific programs, the additional . 
efements of the presence of Federal program fundsAand the percent of target groups were considerations for 
stratifying the school sample. To collect this kind of data, a precanvass of districts was conducted! 

The primary purpos^ of the precanvass was to provide up-to-d^e infoimation about specific schools so that the 
pretest would be sent to the schools that had the Federal programs of inteiest as well as the target groups needed for ' 
the supplementary sampling. ' ^ ' , ' | 

/ In February 1974, the contractor telephpned each 6f the nine participating States and spoke with tlje SSES ^ 
survey coordinator. After agreeing on the districts within a State that werej to participate in the pretest, a number of 
general questions were asked that dealt <vith survey logistics and' the bes!t source in the State or LEA for specific 
information about individual schools. .With the exception of Florida, each of |he States indicated that the 
information resided in State files and that the State education agency (SEA) was the best source of information. In 
late FebruaBy 1974,^^prQcanvass fom>y4nd instructions were sent to the SEA offices in the nine States. ' 
• The precanvass forms were in three parts. The first part contained two pages of specific instructions indicating 
;how the respondents were to complete the infolrmation on the fomis provided. The second part was » computer 
iisting generated from the Public School Unive^^ File that was current as of October 1971 (the latest available in 
February 1974). Respondents were asked to make corrections and/or additions to! the printout concerning ea(^ of 
tjjie elementary schopls listed for a given disftrict. The third part was called the| Program Informatiorf Form and 
respondents were asfk:ed' to indicate, by school, the following types of information: 

*^l/^ V\^ether the school should be excluded from th^ . * 

1^. Presence of programs funded by thff Federal sources listed. , \ 
. 3. Approximate percent of listed pupj:! target groups. 

... ^. ' . '"^ ■ ' ■ % . , 

Samples of the instructions and forms appear at the end of this appendix. * . 

Respondents, were asked to return data by March 8, 1974, to facilitate the school selection process. ' 

> RETURN OF PRECANVASS DATA 



By Marqh 8, 1974, no State had responded to the precanvass request. And, in fact, by March 25, only three States 
had mailed in their forms. Telephone calls made or received by the contractor and OE personnel during the 
intervening period revealed that in several States the data requested were not available at the State office— even 
though all of the States but one had originally indicated that the precanvass information could be readily provided 
. frbm existing State records. Consequently, the forms were either mailed to the individual LEA's or phone calls had 
to pe made for the missing in fomiation (generally percent of target groups), thus causing the delays. In addition, two 
States sent wriitten notes with the precanvass forms stating that they were unhappy with the quality of the data they 
had been able to supply at the State leveL In at least two other States, the problem was only too apparent from the 
indicc'uracy of incompleteness of the returned forms. One State in particular had apparently supplied numbers 
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instead of percentages for the taTger^^5ps.^ach LEA then , had to be called to ascertaii>|)vhether the hgures 
provided by the State office were indeed percents or numbers,>When it was discovered thcy/weife numbers, percent 
conversions had to be made. This particular State did not retufif the precanvass materials uadl May 13. 

Some States complained, howevet, that the turnaround titne allowed them was too short, aijd that was certainly 
part, of the problem. IXiring the State interviews, which were conducted as p^ of thy validation field work, most 
States felt that 2 weelts to a monm would be necessary to complete the precanvass at the State leivel for the full-scale 
survey. This length of time is ifeibded because, although most States maintaih compfiterized'rlBcords on the type'of 
information requested- by th/precanvass, in many injftances it is not received from the districts,' processed, and put 
onto tape until after several months of the new schopl year have passed. Several^States cited this as an argument for 
conducting the prec^vas3''at the LEA level: at the tirae of the year the SSES would need precanViiss information (in 
the fall preceding ihi spring mailout), the LEA's a^still the best current source. 

School sahipling ;pi^cedures^e'r^ begun on Mitch, 29, I9i4, in spite of the fact that three of the nine States had 
not yet responded \d the precanvass. As a result, school sampling in two States had to be done on the basis of the 
2-year-old Public School Universe, File information concerijiing grade span and enrolhnent. (The third State had 
volunteered for ms^rict-only participation ar^'did not represent a school sampling consideration.) This was to have 
repercussions in both of these Statest In o'p^'f State (which did not return its precanvass materials until mid-April), a 

• school i^clu()ed in the sample had closed,'^ the ^ther State (which did not returrf-its forms until mid-May), one 
school had /been demolished to jnake way|^br a new highway. Another had dropped its elementary enrollment and 
now served only grades 7 and 8, therefoi^jfemoving it from the-^ico'pe of an elementary school survey. Since none of 
these pi^blems were uncovered until th? Validation field w6rk, three schools were lost from the sample-a situation 

' that njight have been avoided with acc^t^te precanvass information. The implications for the full-scale survey should 

not viewed lightly in that there Jjrfll be i substantial inc|f^ase in the number of districts and schools involved. 

Acidulate and current precanvass data are vital if school sampling procedures are to be conducted with any measure 

qf reliability! | /J 

/ ■■' - ' J ^ • • ' ' - 

/ RESULTS of" THE PRfeCANV ASS OPERATION / 



While the pretest of the precanvass was carried out With some difficulty, it did in fact achieve a more than 
reasonable rate of completion lind the necessary data was received from most States in time to be used in the manner 
for which it w^Jnfended; (The latter woujd not have been the case, however, had the pretest mailing not been 
delayed by the 0MB qlearanee'procedure.^The States took much longer in responding than was expected and, in our 
opinion, longer than should have b(*n nepessary.) * • * • 

The actual selectiori of schools foi;/he pretest was not a strict, stratified, random sampling of school within 
participating districts. This kind of seJ6ction was not possible because of the limited size of the pretest sgmple. The 
information obtained fron^ the precanvass was used in selecting schools* however, to make sure that enough schools 
of each type were selected for testing all aspects of each questionnaire and of the survey operations. 

The following sections prescnt some of the major findings of the precanvass operations, sefected summaries of thQ 
data, and general conclusions. / r 

Data Presentation / 

1. The distribution of X^ie 1,437 schools in selectetTSistric^} in the nine States surveyed is as follows: The number 
• of schools for each'State will vary throughout this presentation of results because: (a) some coui)ts are missing 
for the 101 schools that were selected from the master^tape for the pretest mailout, (b) other counts of 
schools will change according to the number of schools added or deleted in each State, and (c) where States 
did not provide data for some schools, the base number of schools will change. 



State 


Number of Schools 


State ^ 
Minnesota 


Number of Schools 


Arizona 


31 


157 


Colorado 


91 


New Jersey 


69 


Rorida 


307 


Texas 


463 


Kentucky 


69 


Virginia 


124 


Michigan 


126 


. Total 


1,437 



ERIC 



98 

99 



/ 



2. The following statistics refl^ reported changes or updates of information supplied from the Public School 
Universe^le. 

a; Nuttiber of schools that were reported as being added to or deleted from the computer listing supplied. 



State 

Arizona 

Colorado 

Rorida 

Kentucky 

Miftnesota 

New Jersey .... 

Texas 

Virginia . 



Total 



Percent 

b. Number of schools reported as having -changes in grade span. 



Original 


Added' 


Deleted 


Gross 
change 


Perrent 
changed 


Net 
change 


29 


4 


2 


6 


21 


+2 


8? 


. 3 


0 


3 


3 


+3 


345 


0 


• 38 


is 


1 1 


-38 


80 


. 6 


17 


23 


29 


-11 


158 


0 


1 


1 


0.6 


-1 


67 


4 


2 


ff 


9 


+2 


468 


■ 12 


17 


29 


6 


-5 


123 


_[ 


_0 


1 


0.8 


+1 


1,358 


30 


77 


107 




>-47 




2 


6 


8 




-3 



State/Schools 


Number with change 


Percent 


Arizona/22 


3 


• 14 


Colorado/76 


4 


5 


norida/292 


72 ^ 


25 


Kentucky./5 1 


31 


31 


Michigan/120 


• 12 


10 


Minnesota/1 57 


4 . 


3 


New Jersey /5 5 


8 


15 


Texas/448 


208 


46 


Virginia/ 124 


5 


_4 


Total /1, 34 5 


347 


•26 


umber of schools reported as having changes in enrollment^f 10 percent 


State/Schools 


Number witli change ^ 


Percent 


Ari2ona/22 


10 


. 45 


Colorado/76 


•53 


70 


Florida/292 


287 


98 


Kentucky /51 


46 


90 


Michigan /1 20 


53 


44 


Minnesota/1 57 


9 


6 


New Jer5ey/55 


42 


76 


Texas /44 8 


426 


95 


Virginia/124 


3 


_2 ■ 


Total/1,345 


929 


69 
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d. Number of schools with reported change in the presence of programs for the handicapped. 



State 



Added 



Deleted 



Total changed 



Colorado* 
Rorida 
Kentucky 
Michigan , 
New Jersey 
Virginia 

Total 



38 
53 
10 
5 
5 
6 

117 



I 

64 
7 
3 
3 
0 



39 \^ 
117 

v 

8 

195 y 



3. Schools for which/^tates failed to provide Federal funding and target grotip>^ information. 



- State/Schools 

Michigan /1 26 
Texas/463 

7 - 
Total sample/1,437 



Schools not reported 

54 
15 

69 



4, Schools that were reported as having no Federal funding. 



/ State /Schools 

/ 

^Colorado/91 
--^.aQrida/307 * • 
Kentucky /69 
lVlichjgan/.^26^ ^ 
New Jersey /69 
Virgii)ia/i24 

y 

Total sample/1,437 



No Federal funding 



/ 



51 
I 

II 
24 
25 
46 

158 



Percent 

43 
3 

5. 



''^^rcent 

' 56^ 
0.3 
16 

19 - 
36 " 
37_ 

II 



5. The following numbers and percentages of schools were reported to have programs funded by the Federal 
sources listed. , 



Stato/Sctiools 






Nfigrant 


ESEA III 


ESEA VII / 




EMA,B 




No. 


. Percent 


•No. 


Percent 


No. 


Percent 


No. 


Porcei|t 


No. 


Percent 


Arizona/3 1 


14 


45. 


5 


16 


17 


54 


0 


/ 


0 




ColQntdo/91 


34 


37 


9 


10 \ 


V 10 


11 


0 




0 




Florida/307 


86 


28 


111 


36 


^162 


53 


2 


0.^5 


0 




Kcntucky/69 


54 


78 


5 


7 




40 


0 




4 


6 


Mlchigan/126 


38 


30 


16 


13 


24 " 


19 


5 




0 




Minnosota/157 


57 


36 


1 


0.64 


' 41 


^26 


0 


x 


0 




New Jcrs6y/69 


32 


46 


12 


17 


5 


7 


3 


4 


1 


l.'S 


Tcxa8/463 


233 


50 


85 


18 


1 


0.2 ^ 


0 




0 




Virginia/ 124 


60 


48 


12 


9 


7 


5 


0 




27 


21 


















'-11 






Total/1.437 


608 


42 


256 


18 


295 


20 


10 


/ 0.69 


32 


2 



101 

100 
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6. The following is a distribution by pupil target group of the number of reported schodls (by State) having one 
or more percent for each target population. • \ 


State/Schools 


' Migrant 


Bilingual 


Indian 


Handicapped 


Arizona/ 31 

* Coloradory91 
FloridafStW 
Kcntucky/69 
Michigan/li6 
Minncaota/IST". 
New Jer9cy/69 
- Texai/463 
Virginla/124 


i 

2 

N/R 
- 4 
16 
N/R 

N/it 

- 77 
6 


3 
5 

N/R 
N/R 

35 
N/R 

52 

5 * 
3 


3 
32 
N/R 
N^ 

ft 
N/R 
N/R 

46 
4 


2 
67 
N/R 
40 

* y 31 
^ N/R 
V ' 24 
N/R 
51 


Total/ 1,437 


106 ^ 


103 


96 




Percent ^ 


7,37 


7.16 


6.7 


IS 


* : 

(N/R means no roiponie provided.) 








7. Schools excluded from the pretest (as indicated in the precanvass form by respondents) occurred in the foIlo(v- 
ing States. 


State/Schools ' 


Schools excluded 


Percent 






Kentucky/69 
Michigan/1 26 
Virginia/124 


37 
4 
2 


54 
2 
3 






Total sample/ 1 ,437 V 


43 


3 







8. As part of the validation stud^, interviews were conducted with reprdentatives of the nine SEA's. (Responses 
^ shown are expre^d as a percent of those responding to each question.) 

(1) Arc^^£m4he-who^um the information collections for the precanvass? 
78% « YES 

(2) Did you have much difficulty providing requested inforrffation? 
57% = YES 

(3) Did you have all the nccessyy information on hand in the Slate office? 
57% = No 

(4) Did you find it necessary to gel some or all information from the districts? 
50%= YES 

(5) Did you contact any schools to obtain some or all of the information requested? 
I7% = YI^S 
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(6) What do you think is the best place for OE to seek information? 



37% = Indicated that the school districts would be the best source 
Statistical Summary 

1 . Of the 1 ,437 schools reported on : 



a. Information Information on Federal funding and target groups was not provided for 69 schools, or 5 
percent. However, the schools not reported on occurred entirely in two States. 

b. 1 58 schools (or 1 1 percent) were reported as receiving no funding from the Federal sources listed. It should 
' be noted that 97 of th ese 158 schools (61 percent) were from only two States. 

c. Only 3 percent of the schools were indicatpd for exclusion from "tfie pretest. This figure is further 
encouraging in tliat 84 perceat of these schools (37 of 44) were from one State. 

2. Percent of schools receiving Federal funds in the districts selectedV-^'"' 

V 

ESEA I 42% . . 

ESEAIII 20% ^ ^ ^ 



ESEA I Migrant 18% / . \ ^ 

EHA,partB 2%* ' ^ \ 

— ESEA VII . 0.69% — , K 

3. Schools were counted that indicated at least I percent of a particular pujJif target group.' As a result of sucIk 
counts, pup'i! target groups are shown to exist in the following^ercentages of schools in the districts selected: 

Handicapped 14.96% 

Migrant 7.37K% 

BOlngual 7.16% 

Indian 6.70% 

a. The updated data reflected a reported addition or deletion of 107 Schools. Thus the file did not have 
current information regarding the existence of 8 percent ofiho schools in the districts selected. 

b. Indication of the grade span tauglit in each school .^rfschanged for 332 schools, or 25 percent. 

c. SEA'S were instructed to change the Enrollment figU^jMi^tcdJor each school if it was off by more than 10 
percent. Tlie cnrollmdnt figures were changed for 92^ (or 69 pefcent) of the schools listed. 

d. The listing Showed whetlter or not a school had ^ prggranTlor the handicapped. "No" was changed to 
"Yes" for 117 schools and the reverse took place foikS^hools. Thus, tliis indicator was not up-to-date for 
195 (or 14 percent) of the schools. • 

'■■■'■/ • / 

General Conclusionis } ^'^ / 

1. Response , / * 

a. The rc^onSe to the precanvass was generally good, in that all nine States did respond. However, this was 
not i^omplished Without great effort. In addition, if the original schedule had been followed, the inputs 




•Total pofccnta^b of EH A, part D, programs Is probably somewhat inflated ilncc 84 percent of the schools (27 of 32) were from one 
8U.C - , 

/ ■ '02 



from a number of States would have been too late to be used. As it was, one State did not respond in time 
and only partial response was received in time from another State. • . . ' ' * 

b. In t)ie final analysis, information was provided for 95 percent of the schools for which it was requested. In 
addition, the missing 5 percent was from only two States. The other seven States provided complete 
information. 

c. It would seem that the precanvass achieved its purpose in acquiring funding an^ pupil target, group 
informatipn for 95 percent Of the sch(jt)ols iivthe selected distriSts^ This would certainly be a more than 
adequate response for purposes of stratifying the school samples. 

Information from the Public School Universe File 

I 

a. A precanvass of sampled districts appears to be necessary for the full-scale survey if only for the purpose of 
updating information from OE files necessary for school stratification and for survey controls. 

(1) Eight percent of the schools on the final updated fil6 would have been-in error-~eitherasa result of no 
longer being in existence or not having been added to the file. ^ * 

»• . 

(2) The grade spans served was in error for 25 percbot of the sch\^ols. Grade span information is quite 
essential for proper sample selection to ensure that all types of school^ are included with equal 
probability and that only schools with grades prekindorgarten through 6 are included. (There were 
problems with previous Elementary SchdDls Surveys (ESS^s) with regard to including schools in the 
sample that no longer had the required grades,) Problems of this sort will undoubtedly introduce 

^ additional concerns about response weighting. ^ 

(3) The nine States precanvassed indic^ated that the enrollment figures provided were wrong by at least 10 
percent for 69 percent of the schools listed. TRe implications of this finding are obvious. 

(4) The indicator for presence of a program for the handicapped was in error for nearly 15 percent of the 
schools. Use of this indicator, without updating, for school stratification would no doubt present 
problems. 

b. The information available on the Public School Universe File at the time of the precanvass (February 1974) 

was current as of October 1971. The problems experienced with the data on this file are possibly twofold: 

the age of the data and errors in creating the file. How much the data on file was incorrect as a result of 

either of these factors is hard to evaluate. However, it seems reasonable to presume that to moke proper use 

of this file In sample selection, the information contained on it should be current as of the school year in 

question or at least not more than 1 year old. 
> 

Precanvaks-at-tliaSEALevel , 

a. Interviews were cortducted^th representatives of the nine States involved in the precanvass. In seven 
States, tlic'interviewdf* was me person responsible for the precanvass. In the other two States, the person 
responsible for the precanvass was not available for the interview. (One of these two was the only State 
which indicated prior to the precanvass that the predanvass could and should be done directly with the 
districts involved. That State later agreed somewhat reluctantly to provide the information from the SEA.) 
The responses of these seven States showed: 

• 57 percent indicated they had difficulty providing the requestdd information and further indicated that 
they did not have all the information in th.e SEA office. 

• 50 percent indicated that they found it necessary to obtain some of thc^ information from the LEA's. 
Further, l7 percent indicated that they got information dirc^ctly froip the schools. / 
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• 37 percent indicated that the LEA's would be the best Source for the information. 

b. It is no wonder, then, that the precanvass tobk much lorjgcr. than expected. There is no doubt that this 
operation will take much longer and will be a considerable^ burden for the States if conduct^ed at the SEA 

^ ^ level for the fill-scale survey. The full-scale survey would necessitate precanvassing upwards of 800 districts. 
The lea's could provide the information^much more quickly and efficiently. 4 

c" Provision of target group percentages: 

(1) The percentages of schools reporting 1 or more percent of the four pupil target groups seems quite low. 
Among the reasons for this are: ^ 



• two States reported no pupil tirgEt group percentage' at all, 

• two States did not report any sdhools for two of the categories, and 

• many States provided figures for only some of the schools. 

(2) It is likely that a number of States had great difficulty providing this infpiTifation at the State level 
(even though ^ight of the nine States polled prior to the precanvass^ifmicated that they could, and 
preferred to, provide the information ^fom the SEA). This Jrtay be another cause for the low 
percentages. Perhaps one more inference could be drawn frprri these low percentages: the numbers of 
these target pupils are low and thus do not show in tlit figures for very many schools. If there is, 

/ indeed, any validity to this supposition, it further ad^s weight to the argument that the school sample 
must be stratified by program and pupil target ^oup to achieve adequate representation for those 
programs and groups that have been so poorly represented in the data from previous ESS's. 

(3) The statistics on percent of handicapped pupils are cited as a case in point. ' 

• It is highly likely that many more than 15 percent of the schools have I or more percent 
handicapped pupils. (In fact, if this is not the case, then the data on changes of status [1951 of 
programs for the handicapped is much more significant than pneViously assumed.) 

• Three States did not give information on the percentage ypr handicapped pupils for any of their ' 
schools. And yet, handicapped data was the primq^^ example cited by the CEIS Surveys 
Subcommittee in August 1973 of data that could be easily provided-by the SEA. 

Further argument for eliciting the necessary information through ths^precanvass direfctly from the school 
districts Can be made by examining the following sample of problems resulting from the information 
provided hy the precanvass: 



• A scltool in one States was reported»as having no ESE/l tjtic VII program and no program for the 
handicapped. Tlie LEA office indicated that this school had botlrprograms. LEA and school records 
confirmed this. ^ 

• In at least one instance, the SEA coordinator handed the precanvass materials to the LEA coordinator 
for Completion. 

• Tw0 schools in one district were indicated as having percentages of Indians in their school populations. 
Both schools were subsequently picked for supplementary sampling of Indians. Neither school, according 
to the LEA, had fl/2>' Indians. % 

• Various other errant indicators of the presence or absence of Federal programs were also seen. 

4. Summary 

a. The precanvass is an essential part of presurvey operations because of the need for updated data on schools 
as well as additional information to properly stratify a sample of schools. Tlie stratification is necessary to 
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epresent Federal programs and pupil target groups that have been inadequately represented in 



ion from the Public School Universe File, used in the stratification and selection of thfe school 
le in the past, is inadequate at best. * « . 

lie States provided the precanvass information with considerable difficulty. The time required was longer 
than the survey schedule'would normally allow. There were significant gaps in the data and some serious 
errors were seen-«ven without any attempt to verify the data provided. 

It is recomrnended that a precanvass be conducted early in the ^1974-75 school year and the information be 
solicited from the samplad districts: either by mailing to the districts directly or by having the SEA's 
distribute 1;ne precanvass materials to the districts, with provision for return directly the contractor*. 
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INSTRUCTIONS FOR COMPLETING THE PRECANVASS SURVEY 



There arc two separate attachirtents included here: 

A. A computer listing 

B. The Program Information Form ^ <r ^ 
aNmE COMPUTER LISTING 

The only, schools listed are those that have any or all of the grades Prekindergarten through &. This listing is based 
on infomration provided for the Elementary and Secondary General Information System (ELSEGIS) for the 
"1971-72 school year and may need some updating. 

We would like you to make any necessary corrections to the listing. 

PLEASE MAKE CORRECTIONS IN THE' FOLLOWING WAY: DRAW ONE LINE THROUGH THE 
INCORRECT DATA ITEM my^RlNT THE CORRECT INFORMATION ABOVE THE INCORRECT IN- 
FORMATION. 

1. USOE School Code: If you are familiar with your USOE School Codes and they are in error, please make 
corrections. 

2. Correct the name of any schools that are in error. (It is not necessary to add words such as "Elementary^ or 
"School" if they are missing.) ^ . 

3. At the bottom of the page add any schools having grades Prekindergarten through 6 that are not on the list. 

4. Check grade span for each school and correct if in error. 

5. If the total number of pupils is wrong by more than 10 percent for any school, please make that correction* 
(Exact numbers are not expected; your best estimate will suffice.) 

6. "*^ the column marked "HDGP PROG" indicate whether or not an individual school has any program for the 

handicapped that physically operates in that school. Please correct the •'Yes" and '"No" designaUons if they 
are incorrect. 

You have completed the first part of thepxecanvass for the pretest of the SSES. 



THE PROGRAM INFORMATIONFORM 

1, In column 1, write dovim all the school codes as they appear on the correct computer listing. 

2. In column 2A, write the name of each school in this jiistrict in the order it appears on the computer listing, 
Including those you have added, (It is not necessary to write the words "Elementary School'' behind each 
name-for example: Palmdale, Valley View, Sheraton, etc., will sufficej 
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Sanln column 2B, make an "X" beside any school that you feel should be excluded fxbi^this pretest of the 
, Statistical Survey of Elementary Schools (SSES). (Ofi the reverse side of the Prograrnin formation Form, 
please state briefly the reason for excluding this school.) 

3b. In column 3, make an **X'* beside any school that does not have any programs funded by the following 
sources; 

• ESEA title 1 (Qiildren from low-income areas and neglect^or delinquent children) 

• ESEA title I Migrant 

• ESEA title III 

• ESEA title VII (Bilingual) 
EHA, part.B (Education for the Handicapped A/T, part B) 

X ' ■ i 

4. In column 4, indicate for each school (othei^an those you marked in column 3) the presence of programs 

funded by any ^f the five Federal sources quaking an "X" under "YES" or "NO" for each program. 

5, In column 5, estimate for each school tn/approximate percentage oT^achof the listed pupil target groups. If 
none, write a "0" in the appropriate cyiumn(s). 

Use the following guidelines as defi/tions for these target groups. 

a. Migrant pupil: a child of a migmm agricultural worker who haS moved with his family from^onc LEA to 
another during the past year in ord^rvthat a parent or other mehiber ofhis family might secure temporary 
or seasonal employment. 

b. Bilingual pupil: any pupil whose dominanf language is not English. 




c. Indian: any pupil who has been classified as a Native AmeTlcafr&idian by local guidelines. 

d. Handicapped pupil: any pupil who has been classified a^andlcapped according to State guidelin 
6, Please complete the information requested at the bottoft of the page and sign your name. ^ 





YoM have ^completed the precanvass task. Please enclose m&^^cojja^tflf'computer listing and the program 
information form in tffe prepaid envelope and mail as soon as possible so that the preteit can proceed ort'^schcdule 
and be completed before too late in the school year. We appreciate your cooperation and thank you for the time 
spent on this task. 
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I>IST. CODE NO. 40-16380 
PAGE^JO. 1 



SSES PRETEST LEA PRECANVASS 
DIST. NAME: UNION CITY 



SCpOOL 

'Code 




SCHOOL NMIE 

, EDISON SCH ' ^ 
GILMORE SCH , 

^HUDSON SQH 

^ROBERT W^fkS SCH 
ROOSEVELT SCiy 
WASHINGTON SCH ' 



FEBRUARY 1974 
STATE: NJ 







TOT AT 




COMMUNITY . 


GRADE SPAN 


PUPILS 


PROG. 


ijNION CITY 


KT0 8 


160*7 


NO 


UNION CITY 


KT0 8 


586 


NO 


UNION CITY 


KT0 8 


702 


NO 


UNION CITY 


K TO 6 


1085 


NO 


UNION CITY 


KONLY 


1027 


NO 


UNI0f4CITY 


KT0 6 


, 1028 


NO 
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SSESI>RETEST 
LEA PROGRAM BMFORMATICM^ 



DISTRICT 



District Code 



STATE 





A ^ 




\ 




Federal funding information by school 


Percentage of target groups by school \ 


School 


y- : . 

^ Name 
Elementary school 


Exclude 


No Fed. 
fundmg 


■ESEA I 


ESEA I 
Migrant 


ESEADI 


ESEA VII 


EHA 
PartB 


% * 
Migrant 


% 

Bilingual 


% 
Indian 


% \ 
Handicapped ; 






Yes No 


Xes' 


No 


Yes 


No 


Yes 


No 


Yes 


No 
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Person Completing This Form: 
_ ^ Nanie: _ 



Date 



Tide 



Signature_ 
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X Appendix B 

^\ REVISED QUESTIONNAIRE \ 

^ PREFACE 

BACKGROUNDS^ 

The questions^ontained in this appendix have been produced and revised as a result of a comprehensive process 
' of questionnaire' revisions and updating. The revisions were necessary to reflect initial user needs and changes in 
these nee^s -as a result of program shifts, new legislation, arid shifting emphasis of analysis issues.. In addition, 
changes have been made as a result of a detailed analysis of the spring 1974 protest described earlier in this report. 
The quel'tionnaires contained here must be considered draft questionnaires that would require some additional 
.revision |efore being implemented as part of a full-scale survey. 

The i^mainder of this section presents the following general questionnaire areas that should be addressed prior to 
full-scal4? implementation : 

•/ V • ' ■ •' ■ ' 

( (l)|Logistics ' 

(2) |l>efinitions - - * i. ' 

(3) b Level of respondent • ^ . • 

(4) 1 Numbers versus percents ^ 

(5) Low-income (Questions . » 

(6) ' Racial-ethnic questions 

(7) Other (specify) * : 

(8) Response intervals ^ 

(9) Programs for the handicapped 

(10) Respondent burden , ^ 



LOGISTICS 

The following recommendations are made on the questionnaires^as they affect logistics: ' ^ ' 

• The covers -of the questionnaires should be printed in different colors to facilitate distribation-^nd avoid 
confusion about the various levels of respondents. ^. 

• Instructions, skip patterns, and words needing definition throughout the questionnaires should be printed in 
red. The item analysis generally showed that the respondents did not give these areas the necessary attention; 
This resulted in errors that might have been avoided had instructions, skip patterns, and definitions been in 
red. The use of red printing for the$e areas may reduce these types of errors and facilitate editing arid error 
resolution once the questionnaires are returned. 

• Because of the large volume of pupil and teacher questionnaires planned for the full-scale survey, the 
questionnaires should be prepared for optical scanning to reduce processing time and costs, 

• The School List of Special Programs should be. printed on the cover of the teacher questionnaire so that 
programs applica^e to the specific teachers are identified prior to distribution of the survey materials. 

Ill 
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DEFIN^ONS 



V', ■ <: ■ . . . . ' 

The:.;definitions containea in the questionnaires were, developed using OE handbooks as a basis in conjunction 
with usprs tod contractor staff meetings. These dennitions should be updated vyhere specific recommendations are 
made aSj part of the Item analysis. Among these are: , 

° • C<iilsideration sh9uld be given to defining the term "regular school programs/" 

A^'d "including preschool" to the definition of prekindergart?n. 



• Tl^*. term "presentee training" should be revised to include any training given to a teacher by the school 
district or school before the school year begins. . j ■ ,.. 

• Cppsideration should be given to clarifying the terms "participating" and/ "participants' in Federal programs. 

A copy iif the definitions used for the pretest are included at the end of this pieface. the definitions should be part 
of each qtuestionnaire as a fold-out page in the same manner as they ajjpeared L the pretest 

ii- ■ ■. ■ ■ ■ : 

LEVEL 0]F RESPONDENT 

In the pretest, certain iterris were included at multiple levels of response. Therefore, many items were asked of 
two, thre6 or even four levels of respondents. The item analysis, and the recommendations found therershould be 
reviewed With the objecUve of selecting the optimal respondent. However, it should be remembered Mt some of 
mese Items appear at- more than one respondent level ^ analysis purposes. In cases where there' /ifc no clear 
dommanc^ or need for inclusion to satisfy analysis requirements, the item should be duplicated at tWq' levels and 
reanalyzed:.based upon the vididity and reliability study of the full-scale survey. Chapter 5 of the te;^'t/summarizes 
the analysis of tlie selection of bptimal respondents. j :» umidn^es 

NUMBERS'VERSUS PERCENTS 



In the pretest, certain itertis were tested to determine whether Some data were more accurate or readily available 
usmg number or percent alternatives on different forms of the sam^e question. Generally, number ^^ponses appeared 
to solicit better data. Specific recommendations for each of these items are included in the item analysis Where 
percents are used or whenever the analysis plan for an item necessitates computation of percents U a national basis 
the^percentage basis must be included as an item on the questionnaires. For avaUable base numbers, such as public' 
school enroUment by grade currently collected as part of the ELSEGIS, NGES has the option to merge files for 



LOW-INCOMf QUESTIONS 



The questions on all questionnaires that solicited the definitions of low income were included specifically for the 
^ pretest. A review of the responses to this item should be made to determine whether there is any consistency 
among respondents^regarding the concept of what constitutes low income. If there is. the question>i\sking for the 
defimtion may be dropped for the full-scale survey. If there is not, consideration .pight be given to aadressing the 
entire low-income issue in another manner. y ° ^ 



RACIAL-ETHNIC QUESTIONS 



All racial-ethnic questions should be reviewed to ensure that they reflect'the current terminology used by 0MB 
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OTHER (SPECIFY) / ' , ' 

In many questionnaire litems, closed 'end response alternatives were developed for the pretest ^ver many months 
with the help of the data! users. However, to ensure that the full-scale survey would include all possible responses, 
"Other (specify)" was ad^ed at the end of many pretest itenis. A review of the frequency distributions for each of 
the items'was made to identify high (greater than 10 or 20 percent) response in this category. Wherever "Other 
(specify)" solicited a high response,- recommendations were made in the item analysis regarding reviewing the 
written responses to identify additional response alternatives. This must be done for the teacher and pupil 
questionnaires so they fmay be processed more efficiently-esp^pially given the projected volume for- these 
questionnaires on the full-scale survey. 

, ' » * • 

tePONSE INTERVALS 

For those items soliciting a number response on the pretest pupil and teacher' questionnaires, a line was used for 
fhe respondent <o enter the actual numberi In the early stages of questionnaire return, intervals were established to 
produce frequency distriiutions of the d^ta. The distributions of the pretest 'ciata;were reviewed and recommenda- 
tions about specific intervals were made in the item analysis for every relevant qji^Stion ion the teacher and pupii 
questionnaires. The data requested on these two questionnaires can be easily reduced to categorical responses. Data 
users indicated prior to the pretest that the responses provided by categorical responses were precise 'enough for the 
intended analysis requirements and that these items should be open end only for the ptetest to establish response 
intervals. Consequently, changing these items to categorical responses will not only stfye processing time and money 
but will also provide data that is quite adequate for user needs. \ 

' ■ . . 

PROGRAMS FOR THE HANDICAPPED ^ j 

A review of the results- of-~the. validation study indicateH that the handicapped section contained the greatest 
percentage of eriots (ran^ng from 22 to 39 percent) of any; Section in three of the four questionnaires (schools, 
teacher, and > pupil). The item analysis presents some specific comments for each item and a special paper 
highlighting these problems is available for study at NCES. y\,. 

' Zil^etings were held with^th^ Bur^ of Education for the^ |landicapped (BEH) to discuss the\ pretest results a^^d 
request new data requirement?. BEH mUst obtain information relating to policy issues, for their programs and recerit 
legislative changes. By far the most imp6ft?itit for both of these concerns are the court decisions and Stat^ 
legislation requiring school districts to provide services and education for all handicapped childreh residing in the\^ 
school distri^l.. Lfntil recently, many handicapped children were, receiving services and education from nonpublic ^ 
schools Ov^t-to recent and pending legislation, there is reason to believe that there will be dramatic changes in 
Ihe numbei;^ of harfdicapped children enrolled in the public school system throughout the country. In addition, 
there is an inereq^g emphasis at the Federal, State, and local levels on placing handicapped pupils into regular 
classes (''mainstrea^ng^ Therefore, BEH must have data available to address these and other issues. In an effort 
to provide data for the user, numerous questions were included on all questionnaires to obtain cost information, 
descrfption of services and activities for the handicapped, and, in ioflie cases, fine breakdowns of data by type of 
handicap, grade level, and type of services offered. Unfortunately, the pretest results generally showed that there are 
significant validity problems with these data items as presented in the jtem analysis, which shows the error rate on 
each question. However, an example might give an indication of the magnitude of the problem. 

Question 91 of the pretest district questionnaire contained a matrix requesting data on services offered by type 
of handicapped (286 cells). As a result of recent revisions, the question now includes, three grade distribution 
categories that increase the number of cells to 858. The field validation for this item was^ particularly difficult and 
the field staff felt there were major problems in the validity of this item. In some cases, extremely low response 
rates indicated the failure of r^espondents to answer the question at all. In addition, there may be a great deal of 
echtfi^.and error resolution required for these items if they are left blank. The re\ased question contained in this 
yblume is even more complex and, consequently, respondent burden has increased. - 
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Because of the problems discussed above, a series of meetings should be conducted with BEH representatives to 
approach the problem in a different way. Simply stated, BBH^has a number of pressing policy issues that need hard 
information for decisionmaking. Since this information is not cunently available, BEH has requested a number of 
data items that tfiey believe will satisfy their needs. The contractor suggests that the additional meetings with BEH 
should focus on^^he identification of specific policy issues >s|oppbsed to designing data items. The policy issues may 
thea be\developed to obtain, information requirements, discuss data-collectioa strategy (i.e., mail survey versus 
^ interviews), provide analysis-tationalei and plans, ^nd design^ the specific data items. Only those items that have face 
validity should be included in the spring survey. Whenever jbhere is doubt about the face validity of a question, its 
inclusion on the SSES should be approached cautiously. If' there is doubt about the face validity of many items, 
NCES should consider whether these items are appropriate^\for the SSES or could be better obtained through a 
different vehicle or special study. The accuracy ^required, of specific data items must be addressed within the 
framework of its use for decisionmaking (see chapter 7 of thei text). 



RESPONDENT BURDEN 

♦ ' 

The pretested version of the four-questionnaires elicited a\great deal of useful data from ilie respondents in the 
LEA'S that participated. However, this information was noti obtained without a good deal of effort on the part of 
the respondehts. The front of each questionnaire asked for an estimate of the time required for completion. These 
figures have been summed and averaged for all respondents to a given questionnaire. It is felt that these averages are 
representative of actual respondent burden because th^e LEA's selected for the pretest were reasonably ijpresentative 
of the LEA^s across the Nation. * ^ • 

Using the results of the pretest, the respondent burden for a subsample of the four questionnaires is summarized ^ 
below:' - > 



Questionnaire 


Number 


Average time 
required fdr completing 


Median 


Range 




37 


lOhrs., 34mins. 


8 hrs. 


r hr., 20 mins. to 40 hrs. 


School ....... 


63 


3 hrs., 7 mins. " 


Thrs., 22 mins. 


40 niins. to 12 hrs., 30 mins. 




380 


59 mins. 


Not computed 


10 mjns. to 6 hrs., 55 mins. 


Pupa 


956 


57 mins. 


Not computed 


10 mins. to 6 hrs., 55 mins^ 



The re\ised versions of the questionnaires contained in this volume are likely to change the respondent burden as 
follows: ' ^ 

District Questionnaire: There are 21 fewer item$ in the new version of the district questionnaire than in 
the pretesti version. This represents a JO percent reduction in the total number of items, but is not at all 
indicative of the change in respondent burden. RMC estimates that it will take the average respondent to 
the district questionnaire 9onsiderably longer to complete it because of the complicated items in the 
handicapped section. 

School Questionnaire: The new ^cho6\ questionnaire has five more items than the pretested version for 
an indrd^se of over 8 percent. The principal impact on response burden is likely to be felj in the 
handicapped section where more detailed breakouts of data are now required. • 

Teacher Questionnaire: The new tciacher questionnaire/'has seven additional items, for an increase of over 
1 1 percent. .These changes will not necessarily have any effect on the overall response burden, but there 
is some concern about whether teachers can provide the required information in the program 
participation section without closer interaction between the sampled teachers and the principal's office 
than that which occuned during the ^test. 
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PupU Questionnaire: The new pupil questionnaire has ten fewer items thaii tiie pretested version. This 
reduc^on of over 20 percent is likely to have considerable positive effect c^h the ovefall response burden 
since 8 of the 10 items that were dropped were difficult questions concerning pupil achievement test scores. 
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LIST OF DEFINITIONS 



/. Basic Academic Skills - Any Insiructioo. rogardloss of insifuc- 
llonal methods used, in tho subjoct areas ol mathomaifcs. reading. 
English languago arts, scienco. social studios, and similar subject 
areas. 

2. Educationally Deprived - Any pupils who hovo ncod for spociaf 
educotlongi osslsianco In ordor thai thoir iovcl of oducattonal at- 
tainment may bo ratsod to that appropnaie lor children ot tho^r ago. 
The term lncludo$ pupils who ore handicapped or whoso needs for 
such spoctai educational assistance result Irom poverty, neglect, 
delirv^uency. or cultural or linguistic isolation from the community at 
farge. ~> 

• 

J. Full'Timc Equivalent - The amount of employed time that a 
salaried staff member is available to a school (or LEA) relative to a 
full-time employee It is derlvodoby dividing tne total hours a week 
that an employee is available by the number ol hours a fulMime 
employee is expected to bo in a school (or LEA) Several examples 
6«n best domonstrate this calculation It will bo assumed in^hese " 
examples that a full-time employee is required to work 40 hours a 
week 



)hod?8aw 



A pan-lime teacher who works 20 hod?s a week would be counted 
as a 20 hour&/40 hours 0.5 full-time equivalent tedchor. 



• A psychologist who Is employed full-time by the l^mBk Is only 
available to a school lor 4 hours a week would be counted as a 4 
hourS/40 hours « 0. 1 full-time equivalent on thg School Oues- 
lk)nnaire 

• However, on the District Ouostionn^ire. this same psychologist 
would bo counted as a 1 0 fulMime equivalent.^ 

i 

Handicapped PupHs - Any pupils v/ho have boon classitio'd as 
hanu'icappod according to state guidelines This includes pupils who 
have the following handicaps and for that reasotv could require 
special educational or related servrces: 

a. Trainable Mentally Retarded - Mentally retarded pupils 
who are ca*pable of Ofily very limilod meaninglul achievement in 
the traditional basic academic skills but who are capable of 
profiting Irom programs of tratnitig in'sell-care and simple job or 
vocational skills 

6. Educable MentW^ Retarded - Mentally retarded pupils 
who are educablo >^ the ocademtc. social, and occupotional 
areos even though modorato su|)ervtsion may be necessary 

c. Seriously Emotionally Disturbed - Pupils who havo been 
kjentifiod by profesoionalty qualified personnel as hoving an 
emotloNol handicap-of such a nature and seventy as to require 

; one or more opocinl sorvicpQ. particularly with reference to their 
education, whether or not such services are available, These 
services -~ for conditions such as auhsm. schizophrenia, and 
other psychotic concJiliorrs tncltde but are not limited to 
InsliiuHonal cnre other professional Ireaimenl or care, anti in- 
struction m special clashes lor exceptional pupils on a full- or 
pan-time basns. 

</. Learninfi Disabled - Pupils who have learning dysfunctions 
that limit their ability to learn or function in a re^Uiar educational 
program Those pupMs exhibit disorders in one or more ol (he 
basic processes involved in understanding or using spoken or 
written language Those may be manifested in disorders of 
listening, lalS^'ng. and the cognilive processes involved iR^ad^ 
lng« writing, spelling, or arilhmotic. This includes conditions that 
have boon referred to os perceptual hfindicaps, brain injury, 
minimal bram dysfunclion dyslexia, developmental aphasia, 
etc This term doc5i not inciuue pu^iiis who have loarning prob* 
lomft that are prirrarily the results ol visual, hearing, or motor 
handicapi>. mental retardation, emotional disturbance: or en- 
vironmental disndvantaqe. 



V 



e. Deaf-Blind - Pupils who have both auditory and visual im- 
pairments, the combinatiori pi which causes such severe corrt* 
municattons and other educlitional problems that the individual 
cannot bo property accommodate in special educational pro- 
grams designed solely for the hearing handicappod pupil or for 
the visually handK:apped pupil. 

/. De(^ - Pupils In whom the sense of hearing Is nonfunctional for 
tho purposes of lite (inability to hear connected language with or 
without the use of amplihcatjon). This general group is made up 
of congenttatly deaf and adventitiously deaf. 

g.'iiard of Hearing - Pupils in whom the sense of hearing, 
although defective, Is functional with or without a hearing aid. 
Tho hearing k)ss is generally of a nature and seventy as to 
require one or more special educattonal services. 

/i. Blind - Pupils wfK) are sightless or have severely impaired 
vision, according to the applicable legal dellnitiorv Blindness is 
commonly defined in opHSTOlmo logical te^ms as follows: Having 
central visual acuity of 20/200 or less in the bettor eye with, 
correcting glasses; or having a fietd detect tn which the 
peripheral field is restrtctod to such an extent that the widest 
diameter of the visual field subtends an angular distance no 
greater than 20 degrees. 

i. Partially Seeing - Pupils who have impaired ^vision, usually 
defined according to the Snellen scale as having bettor than 
' 20/200 central visual acuity in tho bottereye after correction b\jt 
with a progressive eye difliculty or with a sOnous limitation in one 
or mdro other visual functions In most casos. pupils having 
between 20/70 and 20/200 central visual acuity are considered 
to have a sdrious visual limitation and to require a special 
educational program 

y. Speecli Impaired - Pupiln who have a severe impairment in 
emitting sounds, words or phrases — including extreme difficul- 
ties such as stuttering, voice disorders, and receptive or Oxpros- 
stvo oral language disorders — that adversely affects tho per- 
formance of a pupil in the regular school program Tnis i& not to 
bo confused with a language (OA£ommunicaCionsj difficulty aris- ' 
Ing from lack of experience with^WJan^^sgo ol instruction^ 
Speech dllticuliles expected to disappear wilh thachild s normal ' 
development ard not included here. ^ 

Ortliopedically Handicapped (Crippled) - Pupils who 
have an onhopodic condition of a tv^e lhat might restrict normal 
opponunlty for education or sell-su|dr|3^. This term is generally 
considered to include pupils have impi^hents caused by )n> 
> genital anomaly (o g., clubfoot, absence of some limb, etc K 
impairments caused by disease (o g , poliomyelitis, bone.tubor- 
culosis. encephalitis, and other neurological involvement that 
may results in conditions sUch as cerebral palsy or epilepsy 
etc.). and impairments caused by accident (o g.. fractures or 
burns that cause contractures, etc.). 

Otiier Healtlt Impaired - Pupils who are handicapped in 
their educational progress because of limited strength.v^itality. 
and altennoss due to chronic health problems such as\ henn 
V condition, tuberculosis, rheumatic 'fever, nephritis, asthma. 
^ alcklo cell anomra. hemophilia, epilepsy, lead poisoning, 
leukemia, diabetes, or other illness. 

Inservice Training - Any prograhi prov^ed<43y the LEA thijl 
contributes to tho professional or occupational growth and compe- 
tence of staff members during the lime of their service to the LEA 
For the purposes of this survey. inserviCQ training should include the 
major emphasis*4Ql the training — not one-hour or less courses, 
seminars, etc , that aro given on a one-time basis only. Sod al^o 
Preservice Training. 
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6, Multigraded Class - A class Including more than one grade and • 
In whicti pupils may bo Identified by grade lavtt; e.g.. a single class 
consisting of sorrio grido 5 and some grade 6 pupils. See also 
Ungraded Class* 

7, Persistent Academic Problems - Any problom in the subject 
areas of mathomallcs. reading. English language arts, scionco. 
social studios, or similar subject areas that has occurred frequently/ 
or that.has been a consistent problem for an extended ponod of lime. 

8, Prekindergarten - A beginning group of class organized to pro- 
vide educational experience for children during the year or years 
preceding kindergarten, which Is a part of the sequential program of 
the elementary school and Is under the direction of a qualified 
teac5br. 

9, Prescrvice Training - Any training provided by the LEA for newly 
hfrod toaclHjrs between the lime they were first hired and the timo 
they actually began leaching at the LEA. Seelnservice Training 
for any trainlrlg that occurs attjor the teacher has begun teaching In 
the LEA. 

10, Special Educational Services for ^he Handicapped - 
Courses of Study designed for pupils who have been classified as 
handicapped according to state guidelines. This coyrso of study 
does not include pupils with reading problems, behavioral prcflSlems. 
or other kinds of learning difficulties, unless those are secijndary 
problems associated with one of the handicapping conditons in 
definition A, 

Handicapped pupils may receive educatianal services In a self* 
contained classroom. In a special school, in a resource roprn. in an 
ltirtOfani7?rachor progcans, oUn-a combination -onru^o activities. 
Also reported In this category are pupils who roceiver'^peech correc- 
tion tho;apy and handicapped pupils whose needs are met in ihd 
regular ciass/oom. 

Related Services for the Handicapped - Special activiUes 
undertaken by a LEA. other ihanJ|hose described above, liial are . 
Inlondodto enable handicapped puptis to acquire the best education 
that they are capable of acquiring. This may include transportation, 
testing, health services, etc. 

//, Target Groups: 

fl. Educationally Deprived - .Any pupils who have need for 
special educational assistance in order that their level of educa- 
tlonal attainment may be raised to that appropriate forchildren of 
their age. The term includes pupils who are handna^ped or 
Whose JTOeds for such special educational assistance result 
from poverty, neglect, delinquency, or cultural or linguistic isola- 
tion from the community at large. 

b. Academically Gifted - Pupils who have been identified by 
profess lono'lly qualified personnel as'being mentally gifted in the 
subject areas of mathematics, reading. English language arts, 
science, social studies, or similar subject areas. 

c. Migrant Pupils (Child of Migrant Agricultural 
Worker) - A child of a migratory agricultural worker who has 
moved with his family from one* LEA to another during the past 
year in order that a parent or other member of his family might 
secure temporary or seasonal emploj^rrionlln an agricultural 
activity. 

• Interstate Migrant As defined for federal compensatory 
education programs, a child who has moved with a parent or 
guardian within the past, yoar across state boundanes in 
order that a parent, guardian.* or other membe^^ of his im- 
mediate family might secure temporary or seasonal em- 
ployment In an agncultural Activity, 



• Intrastate Migrant - As defined for federatcompensalory 
education programs, a child who has moved w»th a parent or, 
QUarsJian within the p^st yf ar across school district bound- 
aries within a Slate In ordei^thal a parent, guardian, or other 
member of his immediate family might secure temporary or 
leasonat employment in an agncultural activity. 

• Settled Out or Five*Year Provisional Migrant - 
Children of migrant agrrcullural workers who have iivod in 
this school district (pr at least five years but are receiving 
services this year as a migrant pupil. 

d. Handicapped Pupils - Any pupil who has boon classified as 
handicapped according to stale guidelines. See also definition 
4. 

e. Neglected or Delinquent Pupils - A delinquent child Is ono 
who has committed a delinquent act and is in need of care or 
rehabliltallonA For the purpose of ESEA Title 1. such a child must 
IH) In the custody of and living in an institution primarrly for 
detinque/it children. 

A neglected child is one whose parents, guardian, or custodian 
, - * have neglected to provide him/her with the support or education 
requlred by law. or medical care necessary to hls*|br well-being, 
or who has been abandoned by parents or cuswdian. For the 
purpose of 6SEA Title I, such a child must boMthe custody of 
and living in an institution primanly for neglecTOd children. 

I2n Teaching Station - Any partof the school (usually but not always 
a classroom) where formal instruction takes place. Open air areas 
are not Included within this definition, suc^as outdoor^ playgrounds, 
baseball fields, etc. 

13, Ungraded Class - A class that Is not organized on the basis of 
grade and has no standa-d grade designation. This includes regular 
classes that h3ve no grade designations and special classes for 
exceptional students that have no grade designations Such a class 
Is likely to contain students of diflerent ages who are frequently 
Identified according to level of porlormance in one or more areas of 
instruction ralhur than Jteiordmg to grade level or age level Un- 
graded classes sometimes are referred to as nongraded." See also 
Multigraded Clasls, ^ — — 

/*/. Welfare - Welfare as used here is Intended to mean the system lor 
providing goods or services to financially indigent or physically 
incapacitated persons or families for the purpose of sustaining a 
minimal IJestyle for an indeterminate period. Eligibility requirements 
arc established by taw or ordinance at the funding sources, which 
may bo either local, state, or federal 
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'GENERAL CHARACTERISTICS 

1. As of October 1, 1974, how many public and non-public elementary schools 
(schools containing any or all of the grades prekindergarten through 6) and 
secondary schools (schools containing any or all of the grades 7 through 12) 
ara located in this school district? (In "b" and "c/' If a school has grades 
that overlap elementary and secondary levels, count that school as bot^f Ster 
mentary and secondary. If there ^re no schools in a category, write "lo,") 

A Public Schools Non-Publlc Schools 



a. 



b. 



c. 



Undupllcated total nunober 

of schools 

Number of elementary schools 
(grades preklndergarten-6) . 
Number of secondary schools 
(grades 7-12) 




2a. On or about October 1, 1974, what was the public school membership M 
prekindergarten, kindergarten through 6, and grades 7 through 12 in thllj 
district? (Include pupils in Special Education and Head Start classes) 



grades 
school 




A 



a. Membership in prekindergarten . . . , . 

b. Membership in grades kindergarten- 6 . 

c. Membership in grades 7-12 ^ . . t 

d. Total membership ^ / . 

b. On or about October 1, 1974, how many teachers (full- and part-time) are em- 
ployed in this school district for grades prekindergarten, kindergarten through 6, 
and grades 7-12? 

Number of Teachers 



a. Prekindergarten ....... 

b. Grades kindergarten- 6 .... 

c. Grades 7^12 

d. Total . 
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3. Which of the following best describes the population of the area in which this 
school district is located?; (Mark one) 

D Large city, 500,000 or more 

□ Large city, 200, 000-500, 000 population 

□ City or suburb, 50,000-200,000 population 

□ City or town, IQ, 000-50, 000 population 

O Rural area or town with less than 10, 000 population 



4. ' Approximately what percent of the pupils in grades prekindergarten through 6 
in this school district are from low-income families? (If none, write "0*') 

^ Percent of pupils in grades prekindergarten-6 



5. \Miat i& your definition of low income? 



6. Approximately how many of the pupils in grades prekindergarten through G In 
this school district are members of the following groups? (If none in a category, 
write "0") 

Number of Pupils 

in Grades 
Preklndergarten-6 

a. American Indian ' \ 

b. Black/Negro 4 

c. Oriental . • . . 

d. Hispanic . ____ 

e. White/ Caucasito^^j^?"^ 

f. Other (Specify); W 

g. Total .... . . . . h . . 



7. Approximately how many of the nmplls in grades prekindergarten through 6 In 
this school district are from hoiiN^ where tlie primary dr dominant language 
Is other tlmn English? (K oione, \mto ''0") 

Number of pupils in gradcb prelihideMgarten-6 
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8a. Have you tested 6th grade reading and received results during this schpojl year 

(1974-75) to date? | / 

□ Yes 

□ No |Go to 8c) 

8b. (If yes to Sa) During the 1974-75 school year, approximately how many 6ih grade 
pupils in this school are reading one or more years below grade lev^el acc<prding 
to current test data based on national norms? 

Number of 6th grade pupil^ reading one or more years below 

grade level (Go to Q. 9) \ " 

8c. (If no to 8a) Have you tested r^^g for any other grade and received results 
during this school year (1974-75) tb date? • i 

□ Yes 

□ . No (Go to Q. 9) 

8d. (If yes to 8c) During the 1974-75 school year, approximately how many pupils 
to the nearest grade below grade 6 for which you do have current test data based 
on national norms are reading one or more years belp^v grade level? (Please 
specify the number of pupils and the grade for which you are reporting) 

u 

^_ Number of pupils reading one or more years below grade level 

Grade for wh^ch you are reporting 



9. Approximately how many of the pupils In grades preklndergarten through 6 In 
this school district would you define as educationally deprived? (If none, write 
"0") • 

. * Number of pupils to grades preklndergarten- 6 
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10. During school year 1974-75, does this school district provide any programs or 
projects funded entirely or partially by the following federal sources? . (Mark 
all that apply) 

Elementary Secondary No 
Program Program • Program 

a. ESEA Ti£le I (children from 
low-income areas, and ^ 
neglected or delinquent 

chUdren) ; □ 

b. ESEA Titie I Migrant 

c. ESEA Titie U aibi^es) ... 

' d. ESEA Title III, SeMon 306 . . □ 

e. ESEA Titie Vn (bilingual) ... □ □ □ 

. f • Education for the Handicapped^, 

Part B (formerly ESEA . ' 

• Title VI) ........ ^. □ 

g. Emergency School Aid Act 

^(ESAA)" □ 

h. / USDA School Food Progij'ams . Q • 

i. yocational Edijcation Amend- ^ 

ments of 1968 i □ 

% 0 Communications Act of 1934, 

Titie in>. Part IV (grants for 

Noncommercial Em cational \; f 

Broadcasting; Corpor^tipn ^'^ \ ^ '^""^ 

for Public Broa(^c^ilsting) ... ............ 

Other federal sources (Specify): 

■ — ^, — — □ 

11, Does this school district ourrentl3re.ti^ployfiionprofes'^nal persomiel (aides) to 
work with children in any of the following^ programs? (Mark all that apply) 



\ 



□ • No nonprofessionals (aides) employed by this school district ' 

□ ESEA Title I 

□. ESEA Titie I Migrant 

□ ESEA Title III, Section 306 

□ ESEA Title vn ' ■ ■ " 

Q ■ , Education for the Handicapped Act, Part B (formerly ESEA Title VI) 
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— : . ^ ^ ^ y^^- . — 

12 • What were the expenditures in grades prekindergarteri through 6 and grades 7 
through 12 for fiscal year 1974 (i, e. , last year) from federal, and state and 
local sources in this school district? Do not include capital outlay and debt ' 
service. Include the 100 through 800 series of accounts, as listed in OE 
Handbook II (1957 Edition). v 



Federal 




a. Total expenditures for grades prekinder^ 

garten-6 •«•••••••♦ ^- • $_ 

b; Tot^ expenditures for grades 7-12 ^ • • $_ 



13 • What was the per pupil expenditure of federal program money for participating 
' pupils in grades preklndergarten through 6 for fiscal year 1974 from the fol- 
iowiiig sources? (If none in a category, write "0") <^ 

, ^^Per Pupil \ 

in Grades J 

. 'i Prekindergarten-G 

a/ ESEA Title I > . , • ^ . . 

* b, ESEA Title I MigTant ^ . . 

c. ESEA Title m. Section 306 . 

d. ESEA Title VII > . . .\ 

e. Educakon for the Handicapped Act, Part B \ * 
(formerly ESEA Title VI) ^. . . . . . . . . \ . . . ■ 

' ' - ' . . B . 

NEEDS ASSESSaiENT 

14. Has a formal (structured) needs assessment applicable t o elementary schools 
been conducted by this school district since the 1963-69 school year? ^ 

□ ' Yes 

□ N9 (Go to Section C, page 7) 
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15. In what year was the last needs assessment conducted for each of the following 
programs (e.g., 1969-70^ etc.)? (if none in a category, write "0") 

J ^ Year Last 
^rNeeds Assessment 
. Conducted 

a. 

c. 

e. 
f. 




ESEA Title I ....... 

ESEA Title III, Section 306 . . .\ . 
ESEA Title VII ........... 

Other federal programs * 

Programs for handicappied children . 
Other special state or local programs 
Other (Specify): 



16. 



Which of the following methods or standards were used as part of the most 
recent ne^eds assessment conducted by this schbol district? (Mark all that 
apply) 



□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 



Cpnference with parents 
Cbnference with pupils 
Staff or teacher conference 
Survey of parents 
Survey of pupils 
Survey of staff or teachers 
Pupil standardized test scores 
Pupil grades 
Pupil IQ scores 
Other (Specify): _ 




17. According to the most recent Jeeds assessment, which of the following were 
identified as major needs in this school district? (Mark all that apply) 

D Curriculum revision 

Improvement. of pupil achieve^ient in the academic areas of: 

' ' □ Reading 

' o ' D Mathematics 

Q English language arts (excluding reading) 

{3 Other academic subject area (Specify): _ 



O Cultural enrichment pro-ams 

O Guidance and counseling services 

□ Health services \ 
n Food services ) 

O Other services (Specify): 

Q More staff 

□ Inservice training 

Q ' New or replacement of equipment, materials, or facilities 

O Evaluation 

□ Dissemimtlon of existing information 
O School llbraary/naedia center 

□ Other (Specify): " ^' 



PROGRAM INFORMATION 

ESEA TITLE I (children from low-income areas, aiid neglected or delinquent children) 

Note: Questions 18-31 deal only with ESEA Title I projects for educationally deprived 
children (from low-income areas), and neglected or delinquent children. ESEA 
Title I Migrant prgjects will' be addres-sed as a separate section , 

18. During school year 1974-75;^ does this school district have any ESEA Title I 
projects for educationally deprived children (from low-income areas), or neg- 
lected or delinquent children? (Do not include ESEA Title I Migrant here)' 



□ Yes ^ 

□ No (CJo to Q. 32, page 10) 
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19, « Do any of these ESEA Title I projects serve pupils In grades prekindergarten 
througji 6? (Po not include ESEA Title I Migrant here) 

. □ Yes - ^ ' ^ 

□ No (Go to Q. 32, page 10) ^ 



20. Do all of these ESEA Title I projects operate only in the summer? 

□ Yes (Go to Q. 27) 

□ No . . 



21. How many elementary public and non-public schools are serving eiigible 

attendance areas for ESEA Title I during school year 1974-75? (If non^ in a 
: category, write ^^0.'^ Do not include ESEA Title I Migrant' here.) ' 

* Number of public elementary schools 

. Number of non-public elementary schools 



22. How were public elementary schools chosen for participation in ESEA lltle I 

projects? (Ma^rk one. Do not include ESEA Title I Migrant here.)* 
• * 

Schools were selected to participate ou the bagiis of a comparison of: 

Q the average nxmiber of pupils from low-incopie families in the school 
district to the niuriber of such pupils in the selected schools 

EU the average percent of pupils from low-income families in the school 
district to the percent of such pupils in the selected schools 

□ Other (Specify): ; • 



23. What was the school district average number (or percent) of pupils from low- 
income families per school used as tlie cutoff point for sel€)ction of ESEA \ 
Title I participant schools? (Provide only one response — either a number or 
a percent. Do not include ESEA Title I Migrant here.) 

^ ■ Number OR ^ Percent of pupils 
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24, During school year 1974-75, how many elementary public and non-public 

schools are participating in ESEA Title I projects? (Do not include schools 
where projects operate only in the summer. If none ip a category, write "0." 
-Do not lnclud(9 ESBA Title I Migrant here.) 

• . Number of public elementary schools 

". Number of nqp-public elementary schools 



25, Duritig school year 1974-75, approximately hov/ many public and non-public 
school pupils in grades prekindergarten through 6 are participating in ESEA 
Title I projects ? (Do not^ include summer sessions. If none in a category, 
write "0. " Do hot include ESEA Title I Migrant here. ) 

* 

Number of public school pupils in grades perkinderg)Eirten«-6 

' Number of non-public school pupils iii grades preldndergarten-6 



26. . Estimate; the amount of revenue this schqol district will spend for all ESEA 
Title I (excluding ESEA Title I Migrant) projects for^^elementary scbools dur- 
ing fiscal year 1974-75. (Include Parts A, B, and C) ' . 



27. Estimate the amount of ^revenue this school, district MlLspiend for ESEA Title I 

sumpier (excluding ESEA>Titfe I Migrant) projects for elementary schools during 
fiscal year 1974-75. '(If none, write "0"), • • 

^ $ 



28. Does this school district currently have a district-wide parent" advisory council 
for elementary-level ESEA Title I projects ? (Do not include ESEA Title 1 
Migrant here) 

□ Yes 

□ No (Go to Q. 32) 



29. Are members of the district wide parent advisory council selected by the 
parents of the children to be served by the elemeutary-level ESEA Title I 
project? (Do not include ESEA Title I Migrant here) 

□ 'Yes 

□ No 
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30. Approximately what percent of the district-wide parent advisory council mem- 
bers are parents of children eligible to be served by ESEA Title I projects or 
are representative of the attendance areas to be included in the ESEA Title I 
project?^ (Mark one. Dp not include ESEA Title I Migrant here.) 

, * □ None 

□ 1-50% 

□ 51-100% 



31. What are the major functions of the district-wide parent advisory council? (Mark 
all that apply* Do not include ESEA Title I Migrant here.) 

LJ Identifying unmet needs through formal needs assessment 

□ Identifying uiunet needs through other means 
D Planning programs to meet identified needs 

□ Review of applications for federal fimds 

□ Evaluation of programs 

□ Implementation or monitoring of programs ^ 4 . , 

□ Other (Specify): ' 



ESEA" TITLE I MIGRANT 

32. During scliool year 1974-75, does this school district have any ESEA Title I 
Migrant projects for pupils in grades preklndergarten through 6? 

□ Yes 

□ No (Go toQ. 38) ^ 



33. During what time of the year do the "ESEA Title I Migrant projects in tliis school 
district operate? (Mark all that apply) 

□ FaU . 

□ Spring ^ " ^ . 

□ Summer 



34. During school year 1974-75, how many elementary public and non-public schools 
are participating In ESEA Title I Migrant projects? (If none in a category, 
write »»0»0 

Number of public elementary schools 



^ Nqmber of non-public elementary scliools 
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35. During school year 1974-75, approximately how many public and non-public school 
pupils in grades prekindergarten through 6 are participating in ESSA Title. I Migrant 
projects in each of the following categories (see definition 11c) ? (If none in a 
category, write "0") ^ ' f 

• ■ ' • ' ■* 

Number of Public Number of Non- 
School Pupils Public School Pupils 
in Grades \ in Grades 
"v. Prektndergartgn-6 Prektndergarten-S 



a. Interstate migrant 

b. Intrspstate migrant 

c. Settled-out or five-year pro- 
visional migrant ...... 



36. Estimate the amount of revenue this school district will spend for all ESE^ 
Title I Migrant projects for elemen^ry schools during school year 1974-75, 
(Include Parts A, B, and C) 



37# Estimate the amount of revenue this school district will spend for ESEA 
Title I Migrant summer projects for elementary schools during school 
year 1974-75. (If nope, write "0") 



ESEA TITLE m, SECTlbN 306 
38. ■ Has this scbcoi district ever received ESEA Title III funds ? 



□ Yes 

O No (Go to Q. 61, page 17) 



39. ■ Is this school district currently participating in an ESEA Title III project 
'funded under the state plan? 



□ Yes 

□ No 
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40. Is this school district currently participating in any ESEA Title HI, Section 306, 
projects? 

□ Yes 

□ No (Go toQ. 57, page 15). 



41. Is the ESEA Title in, Section 306, in this school district a state facilitator 
or developer/demonstration project funded under fiscal year 1974? a 



□ Yes (Go to Q. 61^ page 17) 

□ No 



42, Do any of the ESEA Title Iff, Section 306, projects serve elementary schools? 



□ Yes , ^ 

□ No (Go to Q. 61, page 17) 



\ 

Note: Questions 43-53 ask about the elementary-level ESEA Title III, Section 306, 
project in this school district. If this school district currently liaj3,,more 
than one ESEA Title HI, Section 306, project for ^elementary i^chools in 
operation, answer these questions in term& of the^iproject mos^recently 

* ^ V ■ € 



approved and begun with Section 306 funds. 



43, During school year 1974-75, how many elementary public schools are partlci- 
^pating in th6 ESEA Title-lII, Section 306, project? (If none in a category, 
write "0") 



Number of public elementary schools 



44. Is the elemeijtary-level ESEA Title in. Section 306, project in this school 
district designed specifically for direct pupil participation as opposed to 
teacher training? 

□ Direct pupil participation 

□ Indirect pupil participation (Go to Q. 47) 



45. During school year 1974-75, approximately how many public school pupils In grades 
pre kindergarten through 6 are directly participating in the ESEA Title in, 
' Section 306, project? (If none, write '*o'") 

^ Number of public school pupils in grades prekindergarten-G 
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46* What are the three major criteria used In the selection of pupils in grades 
prekindergarten through 6 for direct participation in the ESEA Title HI, 
Section 306, project in this school district? (Mark the three most Important 
criteria) 

□ No sppcial criterion employed ^ 
O Standardized achievement test scores 

ED Other standardized test scores 

□ Pupil grades 

Q Special needs pf pupils (e«g,, handicaps, problems in discipline, attitude) 
n Teaclier recommendations based on educational needs 
[Zl Teacher recommendations based on other needs 

□ Low income of family 

□ Parent or pupil request * * 

□ Other (Specif): J_ ' 

47. What is the major criterion used in the selection of elementary schools as • 
recipients of ESEA Title in, Section 306, projects or. services? (Mark one) 

□ ^ Individual schools are not selected, or all schools can {iarticipate 
D _ Percent of pupils with low achievement In school 

Q Percent of pupiW with special needs in school (e.g. , handicaps, 
language ol^erthan English, attitude) 

□ Percent of pupils from low-income families in school 

□ Lack of necessary programs from other sources in school 
O Location of school 

□ Enrollment size of school 

□ Percent of minority pupils in school or percent of non-minority pupils , 
, in school 

Q Teacher request 

□ Other (Specify): ■ 



48. Which of the following best describes the major emphasis of the elementary- 
ieVel ESEA Title HI, Section 306, project in thiig school district? (Mark one) 



□ Prekindergarten or early chilcyiood education 

□ Education for the handicapped 

□ Environmental education 

□ Reading instruction 

□ Education of the disadvantaged 

□ Child abuse 

□ Project Information Packages (PIP)i 

□ Other (Specity): 
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49. Which of thfi following groups are direct recipients of elementary-level services 
funded by the ESEA Title in, Section 306, project? (Mark all that apply) 



n School district and school administrators 

n Teachers 

n Guidance personnel » 

D Parents 

D Teacher aides 
Pupils 

1^^ Other (Specify): ■ 



50. For which of the following elementary target groups is the ESEA Title HI, 
Section 306, project specifically designed—either through direct pupil 
participation or teacher training, etc. ? (Mark all that apply) 

□ No specific target group 

D Educationally deprived pupils 

D Academically gifted pupils 

D Migrant pupils 

□ Hnndicapped pupils 

D Neglected or delinquent pupils 
D Pupils from' low-Income families 

□ Pupils from homes where the primary or dominant language is other 
than English 

D Prekindergarten children 

□ Other target group (Specify): [ ' 



51. To what degree does the ESEA Title UI, Section 306, project serve elementary- 
level non-p ublic schools? (Mark all that apply) 

□ Not at all (Go to Q, 54) 

□ Direct participation of non-public school pupils in instructional or 
other services provided for public school pupils 

O Direct participation of non-public school teachers in training provided 

for public school teachers 
0 Provision of materials or portable equipment 

Other (Specify): • 



52. During school year 1974-75, approximately how many non-public school 
pupils in grades prekindergarten through 6 benefit in any' way from the 
servi|2es marked in Q. 51 ? 

Number of nonggMblic school pupils in grades prekindergarten-6 
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53. v^^'How many elementary-l^vel non-public schools are servedpj^e ESEA 
TltljB in, Section 306, project? 

Number of non-public elementary schools 




54. Does this school district currently have an advisory council for elementary- 
level ESEA Title in, Section 306, projects? 



□ Yes 

No (Go to Q. 57) 

_4 



55. of the following participate as laqiembors of the elementary-level 

ES:^^ltle III, Section 306, advisory council? (Mark all that apply) 



CT"' Parents 

O Representatives of community organizations 

Q Teachers 

□ School administrators 

□ School district pcrsoDynel 

D ' Representatives of non-^^ublic schools 

□ Other (Specify): ^ 



56.. What are the major functions ol^tjie ad^^isory council? (Mark all that apply) 

Q Identifying unmet needs through formal needs assessment 

O Identifying unmet needs through other means 

D ^Planning program to m^t identified needs 

Q Review of applications for federal funds 

D Evaluation of programs 

□ Other (Specify): 



57. Has federal funding ever bden discontinued for any elementary-level ESEA 
Title ni. Section 306, , project in this school district because the grant 
period for the project had e;!cpired? 

□ Yes 

□ No (Go to Q. 61, page 17) 
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Note: Questions 58-60 ask about the discontinuation of federal funding for the 
ESEA Title in, Section £^06, project b^ecause of grant expiration. If 
federal funding has been discontinued"^ more than one elementary-level 
ESEA Title HI, Section 306, project, answer these questions tn terms of 
the project that had the highest level of federal funding. 



58. What was the last school year of federaljonding for that project? 
'^^^'^^^^hool year (e.g., 1969-70, etfc.) 

- — — . ^ 

59. After federal funding was dlscjbntlnued, were elementary-level services 
similar to those provided by ^Se ESEA Title in. Section 306, project ever 
continued through state or locaj. support? • ' 

□ Yes , . 

□ No (Goto^ca. 61) • 

60. Compared with the services provided by the federally funded ESEA Title "in, 
Section 306, project, describe the level of seirvlces for elementary schools 
provided by state or local funding during the first year after federal funding 
was discontinued, and describe services provided during school year 1974-75. 
(Mark one for each year. If school year 1974-75 is the first year after federal 
funding ended, mark school year 197*4-75 gnly) 

' ' First Year A£ter School Year 

Federal Funding Ended 1974-75 

a;^ Services were expanded □ □ 

b. Services were continued with approxi- 
mately the same level of staff and 

participants . . ; D * D 

c. Services were continued, but with / 

reduced staff or participants • ....... O □ 

d ^'^j ^ervices were discontinued altogether. . . O □ 
e. Don^t know □ □ 
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ESEA TITLE VII (bUlngual) 

61. During school year 1974-75, how many ESEA title VII piS^cts serve elementary 
schools? (If none, write "0*» and go to Q. 75, page 22) 

Number ESEA Title vn projects for elementary schools 



NOTE: ^estlons 62-68 ask about the ESEA Title VH project in this school district. 
If you have indicated in Q.^/-Sisthat this school district cuirrently has more 
than one ESEA Title vn projectfor elementary schools In operation, answer 
these questions in terms of the project with the highest level of funding for 
. school year 1974-75. 



62. For school year 1974-75, Indicate: (a)Njie number of elementary schools in 
whjch the ESEA Title VH project physicailywOperateg^^ (b) the number of 
elementary schools from which pupils areVelected for participation in the 
ESEA Title yil project (regardless of whethter or Aot the ESEA Title Vn 
project is physically located in that school, aftd^) thewunber of non-public 
elementary schools from which pupils ar6 selected f or paiv^cipation in ESEA 
Title vn projects. (If none in a category, 'write |»0") 

- a. Public elementary schools in which the ESEA T^i},e^l j^iioject, 

physically operates ^ 

. b. Public elementary school$ from whiich pupils are selected for\ 

participation in the ESEA Title vn project 

^ ^ . c. Non-public elementary schools from which pupils are selected 

for participation in the ESEA Title VII projects 
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63, Dyring school year 1974-75, approximately how many public and non-public 
school pupils in grades prekindergarten through 6 are participa^ng in the 
ESEA Title VII project? (If none in a category, write *'0") • 

^ Ninnber of public school pupils in grades prekindergarten-6 

Number of non-public school pupils in grades prekindsrgarten-6 
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64. Indicale below which of the following services are part of the ESEA Title VII 
; project for pupils In grades prekinderg^rten through 6 in this school district 
during school year i 974-7 5,. (Mark all that apply) 

Q Academic Instruction in the basic subject areas of mathematics ^or 
reading 

O Hlstbiy and cultural h^tage associated with dominant language 
" . □ Non-'English language "^s ^ • ^ 

Q Other academic subject areas ' . . 

Vocational subjects 
Q Guidance and counseling 
Q /Testing services 

Q Staff development ■ . , 

Q Community Involvement 

□ Other (Specify): * 

■ ' T — ~ 

Notei 'Questions 65-68 ask about salaried staff members who provide services on 
a regular basis to.the ESEA Title Vn project for pupils in grades preklnder- 
garten through 6. The term "ESEA TltW Vn staff members'' should Include 
all staff—no matter What th^ %ource of funding for their 'salaries — who pro- 
vide services to ESEA- Title VII participants on a regular basis. Interiirtttent 
or unpaid volunteers are to be excluded . If you Indicated in Q. 61 above that 
this school district currently has more than one ESEA Title VII project for 
elementary schools in operation, answer questions 66-68 m terms of the 
project with the highest level of funding for the school year 1974-76. 
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65, In colimin A , indicate the number of salaried staff members (full- and part- 
time) in the ESEA Title VII project for pupils ^in grades prekindergarten 
through 6, (Count each staff member only once. If none in a category, 
. write "0.") ' ^ ; . » 

In column B , indicate the nxmiber of these staff, members who are 'full-time 
. only. (If none in a category, write "0") 

In colimm C , indicate the nimiber of these staff members who are fluent in 
the non-Ei^lish langu^^ge of the ESEA Title vn project. (If none in a category, 
* > write "0") V; ^ ^ 

■ V . ' . ' ■ " ; . -c. -■■ 

^ . I Number of Staff 

A I B ' Members Fluent iTjT 

Total Number of Number of the ESEA Title VII 

. ESEATitle.vn ' Fiill-Time Non-English 

Staff Members Staff Members Languagp 

- • * ■ ■ ■■ ■ I . ^ " ' ^ 

a. Administrators • . | . . . . . . , 

b. Teachers or other i 

professionals ... . . •. ... . . . . 

c. Other salaried staff , 
members (teacher 

^ aides, parents serving ) ' 
as aides, other com- 
munity members, and V 
all paid voluntieers). . • • • • • • ' ' 



66. Has this school district provided in^ervice training for ESEA Title vn staff 
members relevant to the operation of the ESEA Title VII project for pupils in 



grades prekindergarten through 6 

□ Yes 

□ No (Go to Q. 69) 
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67. How many of the following kinds of ESEA Title VII staff members have received 
ln«ervice training relevant to the operation^f the ESE;A Title VII project for 
pupils in grades prekindergarten through 6? (If none in a category, write "0'') 

Number of 

" . ' ' ^ ^EA Titie VII 

, ' Staff Members 

a. Administrators 

b. Teachers or other professionals " 

c. Other salaried staff members (teacher aides, ' 
« parej^ts iserving as aides, other community 

members, and all paid volunteers) . , , 

. — ^ ^ — ' 

68, Which of the following descrijbe the three major emphases of the inservice 
training provided for ESEA Title VE staff? (Marl^ the three most important 
emphases) , 

D Theory and methods relating to bilingual language instruction 
O Techniques in teaching English as a second language (ESL) 

□ Acquisition of the non-English language 
n Relationship of language to self-concept 

□ Understanding of the learning styles of children from different cultures 
Q Methods of assessing skill, language, and knowledge acquisition 

□ Techniques of individualizing instruction 

O Subject area courses for staff members taught in the non- English 
;^language 

n .*<?^tnltural heritage of children whose dominant language is other than 

English / ^ 

n utilization of paraprofessionals and community resources 
n Development of materials 

□ Other (Specify): ■ ' 



^fe9. Does this school district (or any of its schooli^) currently have a parent advisory 
council specifically for elementarji^^level ESEA Title Vn project^? 

□ Yes . ^ . . , ' 

□ No (Go to Q. 72) ' \ 
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70. Which of the following is the primary , criterion used to select elementary- 
level ESEA Title VII parent advisory council members? (Mark one) 

□ Appointed by schoolboard or school officials 

13*^ Electecjl by parents or community representatives 

O Volunteered to serve 

, □ Oth^r^(^acify): . 



71. What are the liiajor functions of the parent advisory council? (Mark all that 
apply) ^ 

O Identifying unnqet needs through formal needs assessment 

□ Identifying unme^ needs thr'ough other means 

□ Planning programs to meej identified needs 

□ Review of applications for federal funds 

□ Evaluation of programs , 

□ Other (Specify): ] 



72. Has federal funding ever been discontinued for any elementary-level ESEA 
Title VII project in this school district because the grant period for the 
project had expired ? 

□ Yes ^ 
O No (Go to Programs for the Handicapped) 



Note: Questions 73 -T^ ask about the discontinuation of federal funding for the ESEA 
Title vn project because of grant expiration. If federal funding has been dis- 
continued for more than one elementary-level ESEA Title VII project, answer 
these questions tn terms of the projfisct that had the highest level of federal 
funding. „ 



73. * After federal funding was discontinued, *^were el^entary-level bilingual 

education services similar to those provided by the ESE^l Title VII project 
ever continued through state or local support? 

□ Yes . . ' 

Q No (Go to Programs for the Handicapped) 
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74. Compared with the services provided by the federally funded ESEA Title VTI 
I project, describe the level of bilingual education services for pupils in grades 
prekindergarten through 6 provided by state or local funding during the school 
year 1974-75, (Mark one) 

Q Services are discontinued altogether ' " ^ 

□ Services are continued, but with reduced staff or participants 

□ Services are continued at^ approximately the same level of staff 
and participants 

n Services are expanded 



Note: Questions 75-102 ask about the handicapped ia.this school dist3?ict. 

"Handicapped*' refers to pupils who have been classified as handicapped 
according to state guidelines. Where questions ask about types of handi- 
capped pupils, report multihandicapped pupils according to their major 
handicapping condition. 



75. Does this LEA provide any special educational or related services to handi- 
capped pupils ? 

□ Yes (Mark all that ajpply) ^ , 



PROGRAMS FOR THE HANDICAPPED 



O Prekindergarten 
Q Kindergarten-6 
□ Grades 7-12 




□ 



No (Go to Section, D, page 37) 



76. 



In your estimation, what is the number of handicapped pupils who need 
special educational or related services but are not receiving them during 
school year 1974-75? 



Number of handicapped pupils in pf ekindergarten 
Number of handicapped pupils in grades kindergarten-6 
Number of handicapped pupils in grades 7-12 



77. 



During school yqar 1974-75, approximately how many handicapped pupils who 
have the following types of handicaps (see definition 4) are provided special 
educational or related services by this school district? (Count each child 
only once according to his major handicapping condition. If none in a cate- 
gory, write "0.*') ^ 

* * Number of Handicapped Pupils 

Kinder- Grades 
' ' Prekindergarten garten-6 7-12 

a. Trainable mentally ret^ded .... 

b» Educable mentally retarded ____ 

c. Seriously emotionally disturbed . . . ^ 

d. Learning disabled 

e. . Deaf-blind ........ \ ... . _J 

f. Deaf 

g. Hard of hearing 

h. Blind. 

i. Partially seeing 

j . . Speech impaired 

k. , Orthopedically handicapped 

K Other health Impaired ^ 

m. Other (Specify): » 
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78. •P.rom the list below, mark the special educational or related services that 
fhls school district makes available to handicapped pupils* (Mark all that 
apply for each grade distribution category) 



a. 

b. 

c. 
d. 
e. 
f. 
g. 
h. 

i; 
j. 

k. 
I. 
in 
n. 



Prekindergarten 





□ 
□ 
□ 
□ 



Diagnostic assessment 

Guidance and counseling specifically for » 
the handicapped .... 

Parent guidance, ahd counseling 

Emotional and social development skills 

Instruction in basic academic skills 

Tutoring: ; 

Language development skills □. 

Speech therapy , □. 

Auditory training □. 

Visual motor training 

Physical therapy □. 

Training in mobility skills ^ □. 

Training in self-care skills CZf • 

Special resources 

(e.g., itinerant tutors, aides, etc.) . . □. 

□. 

D. 

□ 

□ 

...... 



o. Resource room teachers 

p. Learning centers . ^ 

q. Other special services 

(e.g., provision for flexible scheduling) 

r. Pupil transportation 

s. Other (Specify): 



Kinder- 
garten-6 

....... 



□ . 

□ . 

□ . 
□ 
□ 

□ 
□ 
□ 

□ 
□ 
□ 



Grades 
7-12 



□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 
D 
□ 
□ 
□ 

□ 
□ 
□ 

□ 



79# Is there a systematic set of criteria in this school district for recognizing or identi- 
fying handicapped pupils who may need to be referred for diagnostic assessment? 



□ 
□ 



Yes 
No - 
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83. In column A , lndi6ate the total number of salaried staff members (full- and part-time) In this 
school district who provide any special educational or related services to the following types of 
'handicapped pupils in grades prekind^rgarten through 6 > (Staff members may be cpunted more 
than ottce if they teach or provide services to more than one type of handicapped pupil. If none 
in a category, write ''0.") 

» In column B , if possible, provide fuliL-time equivalents for these same staff members. (If none in 
a category, write "0") - • 

A B 
Total Number of 
« Staff Members 
of Grades PreIdndergarten-6 



Number of FuU-lTme 
Equivalent Staff Members 
of Grades Prekindergarten-6 



Fully 
Certificated 



Not Fully 
Certificated 



Fully 
Certificated 



Not Fully 
Certificated 



a. Trainable mentally retarded . . 

b. Educable mentally retarded . . 

c. Seriously emotionally disturbed 

d. Learning disable'd < 

e. Deaf -blind • 

f . Deaf 

g. Hard of hearing • . 

h. Blind' 

i. Partially seeing ....... 

j. Speech impaired , 

k. Orthopedically handicapped . . 

1. Other health impaired .... 

m. Other (Specify): 

11- Total . . . 



84. In column A , indicate tlie total number of salaried staff members (full- and part-time) in this school 
• district who provide special educational or related services to the handicapped in grades preklnder- 
garten through 6 . ,Exclude teachers in regular classes . (Count each staff member only once. If none 
in a category, write **0.'*) 



ERIC 



In column if possible, provide full-time equivalents for these same staff m.nibei^s. (If none ^ 
in a category, write "0") /' 

A B 
Total Number of 
Staff Members 
of Grades Prelcindergarten-6 



Number of FuU-p^ime 
Equivalent Staff Members 
of Grades Prekindergarten-6 



Fully . 
Certificated 



Not Fully 
Certificated 



Fully 
Certificated 



Not Fully 
Certificated 



a. Aj^inistrators 

b. Teachers 

c. Speech therapists • • • • 

d. Psychologists • 

e. Educational diagnosticians 

f . Sodial workers 

g. Physical therapists . . . 
h« Occupational therapists • 
i. Teacher aides and other 

paraprofessionals • ^ • • 

j. Other (Specify): 
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85, How many addition al staff members does this school district plan to emplqy 
during the next school year (1975-76) In budgeted positions who. will provide 
V special educational or related services tq h^dicapped pupils. (]^ none in a 
category, write "0" and go to Q. 87) 

• ' 

Prekindergarten 

\ Kindergarten-6 • • 

^ Grades ,7-12 



86. How many of these do you expect.will be fully certificated by the state to 
teach handicapped pupils? (If none in a category, write "0") 

* Prekindergarten 

Kindergarten-6 . . 

. Grades 7-12 



87. If this school district had sxifficient funds to hire as many additional staff mem- 
bers to provide special educational or related services for the handicapped as it 
would like, how many would be hired?, (If non^ in a category, write "0") 

"\ 

Prekindergarten ; . 

Kindergarten-6 - 

. GradeQ.7T-12 . t 



88. Indicate whether the following activities are part of the services offered in 
this school district's programs for the handicapped. (Mark all that apply in 
each grade distribution category) 

Kinder- Grades 
Prekindergarten garten-6 7-12 



a. 


Workshops for teachers 


... 1 1 • . < 


...... 


.■ □ 


b. 


Tuition assistance for teachers . . 


....... 




. □ 


0. 


Inservice training other than 












....... 


...... 


. □ 


d. 


Programs for parents of the 












....... 




• _□ 


e. 


Information dissemination 










. . . . . 


. . □ . . 


. □ 


f. 


Placement in schools having no 












....... 




. □ 


g* 


Other (Specify): 












....... 




. □ 
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89. How many teachers of grades preklndergarten through 6 in this school district 
are receiving anjr special training during this school year in areas that would 
aid fliem in working wtth the various types of handicapped pupils specified 
below? (Teachers may be counted mpre than once if they are receiving 
traiqing aimed at more than one type of handicapped pupil. If none in a cate- 
gory, write "0") 

Number of Teachers 

Prekindergarten Ktndergarten-S 

a. Trainable mentally retarded . ,\ , . 

b. Educable mentally retarded 

c. Seriously emotionally disturbed • . . . • . . , . 

d. Learning disabled • • • • ; , , . . . ^^^^ 

e. Deaf-blind . • . . . . . 

f. Deaf , , . • • 

g. Hard of hearing • . " ■ 

h. Blind ..... 'ZHZ 

I. Partially seeing . - , . . * . 

j. Speech impaired ^ 

k, Orthopedically handicapped . . . ... . . ..... 

1. Other health impaired 

m. Other (Specify): 
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90. What was the first year of funding for handicapped pupils In grades prekinder- 
"garteh through 6 In this school district (e.g. , 1969-70, etc.)? 

First Year 
of Funding 

a. • Education for the Handicapped Act, Part B 

(formerly ESEA Title VI) 

b. VEA 68 

/'c. ESEA Title I 

d. ESEA Title IE 

e. Other federally funded programs for the 

handicapped ^ . . » 

f. State programs for the handicapped . . . . 

g. Local programs fox; the handicapped 



91, ' Do complementary or Interdependent services exist among programs for 
handicapped pupils in grades prekjndergarten th:nMigh 6 and ESEA Title I, 
IBSEA Title in, ESEA Title VH, or other federaHSrograms, including VEA 68? 

O Yes 

□ Np (Go to Q. 93, page 34) , 
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For each horizontal row below, mark the appropriateA?olunyis for federal 
program acttvitie's that are coordinated with programs for the handicapped 
in grades prekindergarten through 6 in this school district. (Mark all that 
apply) \ 



Programs for the 

handicapped are never 
coordinated with other 
federally funded program"* 
activities. ........... 



b. 



\ c. 



e. 



Common participants in 
needs assessment . . . 



Common information used 
in needs. assessment . . . 

Coordination in the review 
and updating of needs . 

Common participants in 
selection of services . . . 



f. 



g. Staff shared *by more than 
h. 



one service 

Common information 



ysed^o determine 
eligibility of pupils. . . 

Pupils participate in 
more than ooe service. 



m. 



ESEA 
Title I 



Coordination in providing * 

sequential services to « 
follow pupils from 

school to school D . 



Joint efforts in setting 

priorities for nee ds. . . □. 

Common participants in 

pupil selection . . 

joint fAinding D. 

Other forms of 

coordination ' « 

(Specify): [ Q. 



ESEA 
Title III 



.□. 
.□. 
.□. 



ESEA 
Title VII 

....... 



Ail Other 
Federal Programs. 
Including VEA 68 

...... 

...... 

•; 

...... ^ ■ 

...... , 



□ 
□ 

□ 

□ 

□ 

□ 
□ 

□ 
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Sa, To what groups is information about federal, state, or local programs for the 

handicapped usually disseminated within and outside thi^ school district? 
(. (Mark all that apply) 1 



□ None 

□ Within this LEA - • 

□ Other LEAs - . • 

□ Public school staff | 

□ Non-public school staff/private or state instructional staff ' 
O Parents of children receiving services j 

O Parents of children needing services ; 
O . Business an^ industry representatives . ' | 

□ StatjB education agency personnel ( 
O General public 

□ Other (Specify): 



94. Can this sch\)6l district compute per pupil expenditufres for the handicapped 
on the basis ^f type of handicap? 



□ Yes 

□ No 
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85, , Indicate below the source of funds that pays the mpst toward developing, 
initiating, or replicating programs designed to serve handicapped pupils In 
grades prekindergarten through 6 in this school district. ((Mark one on each 
horizontal row) 



Federal Funds State Funds Local Rmds 

a. Develop projects 

b. inittate projects . . • 

*c. Replicate projects • . D □ 



/ 
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- CONTINUED, EXPANDED, OR REPLICATED PROGRAMS 
f FOR THE HANDICAPPED THAT WERE BEGUN 

WITH EHA, PART B FUNDS 

96. Have Education for the Handicapped Act (£HA), Part B (6r ESEA Title VI) 
tmds for handicapped pupils ever been discontinued in this school district 
because the grant period bad expired? ^ / 

■ ■ • 

□ Yes (Mark all that apply) 

Prekindergarten ' ^ 

□ Klndergarten-6 , 

□ Grades 7-12 , .^v 

□ No (Go to Section D, page 37) . 



97. After federal funding was discontinued because of grant expiration, were any 
services for handicapped pupils similar to those provided by EHA, Part B, 
continued through state or local support? 

□ Yes (Mark all that apply) 

D Prekindergarten 
O Kindeii^garten-6 
□ Grades 7-12 

O No (Go to Section D, page 37) 



Note: Questions 08-102 concerii tlie continuation, ''expansion, or replicsatlon of any 
services i3egun with EHA, Part B, funds for elementary pupils by state or 
locally supported programs. 



98. During school year 19M-75, approximately how many 'pupils, by type of 
handicap, do these con^ued, expanded, or replicated programs serve? 
\ (If nbne in a category, write "C") 

Number of Handicapped. Pupils 
* Kinderr Grades 

Prekindergarten garten-6 7-12 



a. Trainable mentally retarded . 

b. Educable mentally retarded 

c. Seriously emotionally disturbed 

d. Learning disabled 

e. Dcaf-blind 

f. Deaf • • • . 

g. Hard of hearing 

h. Blfnd. 

1. Partially seeing 

]. Speccli impaired. ...... 

k. Orthopedically handicapped . • 

1. Other health Impaired . • • . 

m. Other (Specify): 
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99. For the 1974-75 school year, what are the ^enditures from state and local 
sources for continued, expanded, or replicJited progriajns for handicapped 
pupils begun with EHA, Part B, funds ? (If none in a category, yinrite "0") 

$ for $for $for 

Prekindergarten Kindergarten-6 > Grades 7-12- 

a. State sources ..... ___ .... 

b. Local sources . . . . .. .... 

c. Total ......... ..... .... 



100. During the years that EHA, Part B, funding was present in this school district, 
what was the average annual expenditure of EHA, Part B, fui^ for handi- 
capped pupils? (If none in a category^ write "0") ■ ^ 

$ ' Prekindergarten 

$ Kindergarten-6 

$ Grades 7-12 ' " 



101. Without EHA, Part B, funding what i^ the Average annual expenditure for 
continued, expanded, or replicated prograins? (If none in a category, 
write "(>•') , 



% - Prekindergarten 

$ Kindergarten-6 

$ Grades 7-12 




102. Generally speaking, what is the major emphasis of the expenditures for 
continued, expanded, or replicated programs for handicappad pupils? 



(Mark one) 

P Pfanning and development 

LD Operations (e.g., services to pupils, teacher training) 

D Evaluation 

D ^ Parent pr^^grams 

n Dissemination - • , ' 
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EDUCATIONAL BROADCASJING 



103. Does thi^ school district currently hold a license from the Federal Communica- 
tlon^ Commission to operate television, radio, or other transmittal systems 
for Instrucfional purposes ? 

' □ Yes . • 

□ No (Go to Q. 105) 



-L. 



104.. Which of the following does this school district operate? (Mark all that apply 
for each grade distribution) 

' * Prekindergarten-6 Grades 7-12 

a, UHF or. VHF non-commercial television , • 

. ' broadcast station □ 

X b. Translator—UHF or VHF 

c. Instructional Television Fixed Service 

(ITFS). ..... \ □ ■□ 

d. Bro?^dcast FM radio station (10 watts 

or greater), . . □ □ 

e. Campus carrier radio system CH ••••••• EZI 

f. ^ Coaxial cable ("R.F.'') (closed circuit) □ 

g. . Community Antenna Television (CATV) 

(closed circuit) . 

h. Other (Specify): 

: .............. 



105. Does this school district currently have c contract with an agency for television 
or radio programming or transmittal systems ? 

□ Tfes 

□ ^ No (Go to COMMENTS, page 39) 
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106. Which of the following ate covered under this contract? (Mark all that apply) 



CD UHF or VHF non-comnjercial television broadcast station 

□ Translator~UHF or VHF 

D Instructional Television Fixed Service (ITFS) 

□ Broadcast FM radio station (10 watts or greater) 

□ Campus carrier radio system " 
O ^ Coaxial cable rR.F.") (closed circuit) 

□ Community Antenna Television (CATV) (closed circuit) 

□ Other (Specify): ' ^ ; 



107. What is the total amount of fui>ds budgeted for all schools (elementary and 
secondary) during this school year under this contract? (Report to nearest . 
$1,000. If none in a category, write '*0*») . 



Television 
Radio 



108.. Which elementary grade levels are covered by the instructional television or 
radio programming provided under this ,pontract? Indicate equivalent grade 
levels for ungraded or multigraded classes other than for the handicapped. 
(Mark all that apply) 

■» . ' 

If no elementary grades are covered, pheck..here Dand go to COMMENTS, 
page 39. 



Television 

a. Prekindergarten □ 

b. Kindergarten □ 

c. Grade? I ■ . . . □ 

d. Grade 2 ^. . . . □ 

e. Grade 3 . . /. , □ 

f. Grade 4 „ . . □ 

g. Graded □ 

h. Grade 6 D 

i. Ungraded classes for 

the handicapped . . . . 



Radio 

a. Prekindergarten □ 

b. Kindergarten ^ Q 

c. Grade 1 : □ 

d. Grade 2 □ 

e. Grade 3 CD 

f. Grade 4 □ 

g. Grades ; □ 

h. Grqde6 ' □ 

i. Ungraded classes for 

the handicapped . . . . ..□ 
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109, Approximately how many pupils In grades preklndef garten through 6 are 
served under the television and radio contracts ? (If noiie in a category, 
write "0") ' 

Pupils in grades prekindergarten-6 served by television contracts 

. Pupils in grades prekindergarteii-6 served by radio contracts 



COMMENTS 



You have completed this questionnaire. However, if you have any comments you would 
like to make regarding the clarity of the questions, definitions, and instructions used 
in this questionnaire, please use the rest of this pagg. 'THANK YOU FOR YOUR HELP. 
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SCHOOL QUESTIONNAIRE 

» 

's ■ ■ ' 

d 

} 

A* 
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GENERAL SCHOOL CHARACTERISTICS 



1. What grades are taught In this school? Indicate equivalent grade levels for un- 
graded or multlgraded classes other than for the handicapped, (Mark all that 
apply) 

□ Prekindergarten 

□ Kindergarten 

□ Grade 1 . 

□ Grade 2 > . 

□ Grades 

□ Grade 4 

□ Grade 5 ' 

□ Grade 6 

□ Grade-7 

□ Grade 8 * 
Q Grade 9 

□ Grade 10 

□ Grade 11 

□ Grade 12 

□ Ungraded classes for the handicapped 



7 

Note: The next two questions ask about pupil membership on or about October 1. 
Question 2 asks about the current school year (1974-75), while Question 3 
asks about last year (1973-74). 



2a, What was the total pupil membership in this school on or about October 1, 1974? 
(Include pupils in Special Ed&cation and Head Start classes) 



Total number of pupils 



b. What was the total pupil membership in grades prekindergarten through 6 in this 
school on or about October 1, 1974? (Include pupils in Special Education and 
Head Start classes) 

Number of pupUs in grades preklndergarten-6 
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3. Please provide the following information about last year's (1973-74) pupil mem- 
bership. (Include pupUs in Special Education and Head Start classes. If none in 
a category, write "0.") 

A B 
Total PupUs 
deluding Migrants Migrant Pupils Only 



a. Membership during the period 
from approximately October 1, 
1973, through the end of 1973-74 
school year . • . 



b. Number of new pupils entering 
sdhool during the same period 
(cpuntjre-entrles as new 



entries) . 



c. Number of pupils withdrawing 
from school during the same 
period (Include transfers between 
schools and dropouts) ....... 



4. In column A . Indicate how many teachers^ (full- and part-time) of grades pre- 
klndergarten through 6 In this school are members of the following groups. 
(C^SunTsach teacher only once. If none In a category, write "0.*') 

In column B. Indicate how many pupils in grades prekindergarten through 6 in 
this school are membisrs of these groups. (Count each pupil only once. If none 
in a category, write "0. ^») 

, A ' B 

Npaber of Teachers Number of Pupils 
of Grades in Grades 

Prekindergarten-6 Prekindergarten-6 



a. American Indian 



b. 




* .... 

* .... 




c. 




* .... 




* d. 




* • . . . 




e. 


White/Caucasiafa 


..V . . 






Other (Specify):' 
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5. Does this school employ teachers aides or other nonprofessional personnel? 
O Yes 

□ No (Go to 7) 



6. Are any of these personnel specifically assigned to work with children in any of 
the following? (Mark all that apply) 

□ ESEATitlel 

□ ESEA Title I Migrant 

^ ESEA. Title m. Section 306 

□ ESEA Title VH; 

Q Programs for tihe handicapped 



7. Which of the following best describes the population of the area in which this 
school is located? (Alark one) 

\ 

□ Large city, 500, dOO or more 

□ Large city, 200,000-500,000 population 

B City or subjirB, 50,000-200,000 population 
Small city, 10,000-50,000 population 

□ Rural area or town witli less than 10,000 population 



8. Approximately how many of the pupils in this school are from low-Income 
families? (If none, write "0") 

Number of pupils . ■ 



9. What is your definition of low income? » 



10. Approximately how many of the pupils in grades prekindergarten through 6 in 
this schodl are from families where the primary or dominant language is other 
than English? (If none, write "0'») 

Number of pupils 
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11. Approximately how many of the. pupils in grades preklndergarten through 6 in 
this school are from families earning $3,000 or less and are also from families 
where the primary or dominant language is other than English? Of none 
write "0") • . • 

. ' Number of pupils 



12. How would you characterize the English-speakiiig ability of most of the pupUs ' 
In this school? - (Mark one) f 

□ Very limited—conduct little or no conversation in English 

□ Limited— converse in English, but with, more difficulty than would be ex- 
pected of children English-speaking homes 

□ Good— converse in English with little or no difficulty ' - ' ■ 

nZVZ T^l ^'^^^^ ^"^^^^'^ this school year 

(X9<4-7&) to date? 

□ Yes , . 

□ No (Go to 13c) 

13b. (If yes to 1^3a) During the 1974-75 school year, approximately how many 6th ' 
grade pupUs In this school are reading one or more years below grade level 
according to current test data based on national norms ? , C ^ 

, _ Number of 6th grade pupils reading one or more years 

below grade level (Go to Q. 14) 

13c. Of no to 13a) Have you tested reading for any other grade and received results 
during this scHool year (1974-75) to date? 



13 d. 



□ Yes 

□ No (Go to Q. 14) 

■ ^ yes to 13C) During the 1974-75 school year, approximately how marry pupUs 
in the nearest grade below grade 6 for, Which you do have current data based on 
national norms are reading one or more years below grade leve^? (Please 
specify the number of pupils and the gracje for which you are reporting) 

Number of pupils reading one or more years below grade level 

- _. Grade for which you are reporting 
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14. Approximately how many of the. pupils In this school would you define as edu- 
cationally deprived? (If none, write »'0'0 • . 

^ ^^Jtfftlifer of pupils 



15. Are there any bilingual ptojects In this school funded by any sources . other than 
ESEA Title Vn? ' ^ 

' □ . Yes. • 

□ No * 



16. During school year 1974-75, does this school provide any elementary-level 
programs or projects funded entirely or partially by the followi% sources? 
(Before attempting to apswer this question, pleasfe check the District List of 
Special Programs , Mark all that apply. ) . 

□ ESEA Title iv(childfen from low-income areas, and neglected -or 
delinquent, children) 

□ ESEA Titla I Migrant * 

□ . ESEA Title II (libraries) 

' □ ' ESEA Title IE, Section 306 - ' 

□ ESEA Titie Vn (bilingual) 

. \ □ Education for the Handicapped Act, Part B (formerly ESEA Titie VI) 

□ Emergency School Aid Act (feSAA) 

□ USDA School Food Programs ' ^ . 

□ Communications Act of 1934, Titie in. Part IV (Grants for Non- 
eommercial Educational Broadcasting Facilities; Corporation for 
Pv^^lic Broadcasting) 

□ Othdr federal sources / 



17. . During school year 1974-75, doe^ this school provide any elementarj'-level 
programs or projects funded entirely or partially by federal sources that are 
specifically designed to meet the needs of the following target groupg? (Mark 
all that apply ) . . ' 



□ No specific target group 

□ Educationally deprived pupils 

□ Academically gifted pupils 

□ Migrant pupils 

□ Handicapped pupils 

□ Neglected or delinquent pupils 

□ Pupild from low-Income families * 

□ Pupils from homes where the primary or dominant language is other 
than English , \ 

□ Other target group (Specify): ^ 

X. 

■ "7 ' 
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PIICXSRAM INFdRMATION 



'ESEA TITLE I (children from low-Income areAs, and neglected or delinquent children) 

» 

Note: Questions 18-28 d^al on^y with ESEA Title I projects for educationally deprived • 
children (from low-income areas), and neglected or delinquent children. ESEA 
Title I Migrant prdjeots will be addressed as a separate section. 

18. Does this school have an gSEA Title I project for educationally deprived children 
(from low-lncom^ areas), dr neglected oi: .delinquent children? (Do not include 
E;SEA Title t Migrant here) ' 

: □ Yes 

□ No (Go taQ.. 29, page 10) 



Note: If this school currently has more than one ESEA Title I project component in 
operation, answer the remaining questions In this section for all project com- 
ponents. . . 

, ^ a. , 

19. What grades does the ESEA Title I project in this school serve? Indicate equiv- 
alent grade levels for ungraded or multlgraded classes. (Mark all that ap^t^ 
Do not Include PSEA Title I Migrant horq. ) ' 

□ Preklndergarten ^ ' 
D Kindergarten ' 

□ Grade i • - 

□ Grade 2 

□ Grade 3 , 
■ O Grade 4 

□ Grades ' ^ 

□ Grade 6 

□ Grades 7-12 
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Note: If the ESEA Title I project in this school does not serve pupils in^ grades pre- 
klndergarten through>6, check here Q and go to Q. 29, page 10. 



20. Does the ESEA Title I project in ihls school operate only in the slimmer? (Do 
not include. ESEA Title I Migrant here) 

" ;^ ^ / \ . ^ ; 

> . □ Yes (Go to Q, 29, page 10) 
□ No " 



21. During the 1974-75 school year, approximately how many pupils in grades pre- 
kindergarten through 6 participate in the ESEA Title I project in this school? 
(Do not include summer sessions. If you are answering for more than one 
project, do not count kny pupil more than dnce. Do not include ESEA Title I , 
' ^Migrant here.) ^ / / " ^ 

Ntunb^r of pupils in grades preklndergarten-6 ■ ^ ' 



* ■ ■ ■ *' * 

22. Indicate below those, seirvices that are provided as part of the ESEA Title I 

. project in this school for pupils in grades prekindergarten through 6, (Mark 

all that apply. Do not include ESEA Title I Migrant here.*") 

' |_| Reading instruction ^ 

□ Mathematics instruction ^ 4i 
\^ English language arts instruction (excluding reading) 

□ ^ijingual instruction v # „ 

□ English as a second language 

n / other academic instruction ^ ^ ; ^ 

(Specify): ' \ \ ^ 

ri Guidance and counseling services 

□ Attendance and social work services * ^ 

Q Health. services (including medical and dental examinations, instruction 

in health and health cl^re)* " ^ 
n Food services , 

□ Other services (Specify)-^ 
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On what basis w6re pupils selected to participate in the ESEA Title I project? 
Respond only for those project components in which pupils receive instruction 
as part o£ the ^ESEA Title I project. (Mark three in each veribical column. Do 
not include* ESEA Title I Migrant here.) 



Mathematics" Reading 



a. l>upils i^fei^e not selected to « 

^'receive Title I services ^In 

this subject ^rea . . ; , . - 

b. Basis of selecfion is unknc^wn . □ 

c. No special criterion emplbyed ^ □* 

d. Standardized^achi^evement ^ ^ ' 
test,scores 

e. Other standardized test 
SGores : . ! 

f. Pupil grades . .' ' 

g. Special needs of pupils ; 
(e.g.jl handicaps, probl^n?^ , 

<4n^discipline, attitude). .... □ 

-h. Teaser recommendations 

based on ^du,cational needs . ^. □ 

i. Teacher recommendations 

' based on other needs, Q 

j . . Low income of family .... □ 

k. Parent or pupil request . . □ 

1. Other (Specify):^ 

' ' .... □ 



□ 
□ 
□ 



[ English 
Language -^rts" 
(excluding 
reading) 

■ . □ ^ 
. . □ ; 
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24» / How) Is Instruction provided in the ESEA Title I project? (Mark one. Do not 
' ' Include ESEA Title 1 Migrant here. ) . 

Q Puplils receive instruction all day in their regular classrooms as 

part of an ESEA Title I project 
□ Pupils receive instlruction part of the day in their regular class- 

rooms*as part of an ESEA Title I project ' * - > 

' D Pupils go outside of their regular classrooms for part of thet day 

to receive instruction as part of an ESEA Title I project 



25.^ Does this school currently have a parent advisory coimcil for the ESEA Title I " 
project? (Do not include ESEA Title I Migrant here) 



□ Yes 

□ No (Go to Q. 29) 



26jp Are members pf the school-level parent advisory council selected by the parents 
pf the children to be served by the ESEA Title I project? (Do not In^liide ESEA 
Title I Migrant here) « 



□ Yes 

□ No 



-27.. Approximately what percent of the school-lev^l parent advisory council mem- 
bers are parents of children eligible to be seryed by ESEA Title I or are repre- 
sentative of the attendance areas to be included in the ESEA Titje I project? 
(Mark one*. Do not include ESEA Title I Migrant here, y 



O None 
□ ' .^1-50% 
O ^51-100% 
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28. What are the major functions of the school-level parent advisory council? 
(Mark all that apjily. - bo'not Include ESEA Title I Migrant here. ) 



T . n Identifying unmet needs through formal needs assessment 
O Identifying mmiet needs through other means 

Q . , Plaimiig program^ to meet identified needs 

□ Revxe^AT of applications for federal funds 
D V Evaluation of programs 

D Implementation or monitoring of programs 

□ . Other (Specify): " 



ESEA TITDE I MIGRANT 



29. Is there an ESEA Jitle I Migrant project in this school? 
□ Yes 



□ No (Go to Note before Q. 35) 



30. What grades does «ie ESEA Title I Migrant project in this school serve? Indi- 
cate equivalent grade levels f or 4ingraded or multigraded classes. (Mark all 
that apply) ^ 

D Prekindergarten ' " 

D Kindergarten 

□ Grade 1 ^ ' •/ 
D Grade 2 

□ -Grade 3 

□ Grade 4 " ' . 
n Grade 5 

□ Grade 6 



□ Grades 7-12 



Note: If the'ESEA Title I Migrant project in this school does not serve pupils in 
r . grades prekindergkrten through 6, check here □ and go to tlie Note before 
Q. 35. ^ V 

31. During what time of the year does the ESEA Titie I Migrant project in this 
school operate? (Mark ?1I that apply) 




□ FaU 

□ Spring 
D . Summer ' 
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32. During the 1974-75 school year, approximately how many pupils in grades pre- 
klndergarten through 6 participate in the ESEA Title I Migrant project in this 
school? . 

Number of pupils in grades prekindergarten-6 



33. Indicate below those servicesiliat are provided as part of the ESEA Title ! " 
Migrant project In this school for pupils in grades prekindergarten through 6. 
(Mark'all that apply) ' ' 

□ Reading instruction ^ ' 
n Mathematics instn^ction 

□ English language arts instruction (excluding reading) / 

□ Bilingual instruction ^ ' / 

□ English as A second language ^ / 
rn Other academic instruction - / 

• (Specify): ■ .. ' / 

□ Occupational skills familiarization / 

□ Guidance and counseling services 

□ Attendance and social work services 

□ Health services (including medical and dental examiiaations, instruction 
in health and health care) 

n Food services 

□ Clothing 

□ Other services (Specify): , 



34. How is instruction provided in the ESEA Title I Migrant project? (Mark one) 

□ Pupils receive Instruction all day in their regular classrooms as 
part of an ESEA Title 1 Migrant project , 

E] 'Pupils receive instruction part of the day in their regular class- 
rooms as parf^of an ESEA Title I Migrant project 

□ Pupils go outside of their regular classrooms for part of the day \ 
to receive instruction as part of an ESEA Title I Migrant project 



ESEA TITLE ni, SECTION 306 

Note: Please contact your district survey coordinator if you do not know the 
answers to any of questions 35-41. 

s 

35. Is there an ESEA Title in, Section 306, project in this school? 

- - \ □ Yes 

□ No (Go'to Q. 42, page 13) 
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36. Is the ESEA Title m. Section 306, project in this school designed speclfi^Uy 
for direct pupil partidipation as opposed to teacher training? 

□ Direct pupil participation 

□ Indirect pupil participation (Go to Q. 42) 



37/ What grades does th^ ESEA Title IE, Section 306, projept in this school serve? 
Indicate equivalent grade levels for ungraded or multigraded classes. (Mark 
all that apply) 



□ 


Prekindergarten 


□ 


Kindergarten 


□ 


Grade 1 


□ 


Grade 2 


□ 


Grade 3 


□ 


Grade 4* 


□ 


Grade 5 


□ 


Grade 6 


□ 


Grades 7-12 



Note: If the ESEA Title IE, Section 306, project in this school does ni^ s^rve pupils 
' in grades prekindergarten through 6, check here □ and go to Q. 42. 



38. During the 1974-75 school year, approximately how many pupils in grades pre- 
kindergarten through 6 participate in the ESEA Title IH, Section 306, project 
in this school? 

_^ Number of pupils in grades prekindergarten-6 



39. Is your ESEA Title in, Section^306, project specifically designed for the follow- 
ing elementary target groups? (Mark all that apply) . / 

Q No specific target group 

O Educationally deprived pupils 

O Academically gifted pupils 

O Migrant pupils 

Q Handicapped pupils 

O Neglected or delinquent pupils- 

Q Pupils from low-income faniilies 

O Pupils from homes where the primary or dominant language is other 

' than English 

O Prekindergarten children ' ' ^ 

Q Other target group (Specify): 
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40. 



41. 



Which of the following best describes the major emphasis of the ESEA Title in, , 
Section 306jrproject in this school for pupils in grades prekindergarten-through 
6? (Mark one) • " 



□ 
□ 
□ 
□ 
□ 
□ 
□ 
□ 



Preklndergarten or early childhood education 
Education for the^handicapped 
Environmental education 
Reading instruction 
Education of the disadvantaged 
Child abuse 

Project Information Packages (PIP) 
Other (Specify): 




What are the three major criferla lisMIn th©^ selec^pn of pupils in grades pre- 
klndergarten through 6 for direct participag^Jn ilte ESEA Title in, Section - 
306, "J^roject in this school? (Mark'' the three most important criteria) 

^a^lfl^xif selection is unknown 
No sp^qifiiJrCTiterion employed . 
Standardized achievement te^Hscores • 
Other Standardized test scores 
Pupil grades 

Special needs of pupils (e.g. , handicaps, problems in discipline, attitude) 
Teacher recommendations based on educational needs 
Teacher recommendations based on other needs^ 
Low income oLfarttlly ^ 
PafiBnt or pupllTjfequest 

Other (Specify): - — \ 



\ ■ 



^ ESEA^JflTLE vn (bilingual) 
42. Is there an ESEA Title VII project in this school? 

□ Yes ' ^ 

□ No (Go to Note before, Q. 50, page 16) 
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lat grades does the ESEA Title vn project In this school serve? Indicate 
equivalent grade levels for ungraded or multlgraded classes. ^^lark all ^ '~ 
\apply) ^ ' 




□ Prelditd^garten 

□ Klndergarl 

□ Grade 
Grade 2 
Grade 3 

□ Grade 4 

□ Grade 5 ^ 

□ Grade 6 

□ Grades 7-12 



Note: If the ESEA Title VII project In this school does not serve pupils In grades 
preklndergarten through 6, check here □ and go to the Note before Q. 50, 
page 16. ^ 



44. During the 1974-75 schgql year, approximately how many elementary pupils 
participate in the ESEA Title VII project in this school? 

' ^ . J^Jumber of pupils in grades prekindergarten-6 



45. During the 1974-75 school year, approximately how many of the pupils In 

grades preklndergarten through 6 who participate In the ESEA Title VII proj- 
ect in this school speak the following languages as their primary^ jor dominant 
language? (If none in a c^tegoi-y, write/*0'*) 




Number of Pupils 
In Grades Prekindergarten-6 



a. An American4i]L^Jitan language 

b. An Oriental language . . . . 

c. Spanish . . . . ) 

d. English 

e. Other (Specify): 
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46. How would you characterize the English-speaking ability of most of the pupils 
In grades prekindergarten through 6 who participate In the ESEA Title VII 
project in this school? (Mark.one) 

. □ 
□ 



Very limited—conduct little or no conversation In English 
Llmlted~coi\verse In English, but with morerdifflcultytRan would be 
expected of children from "English -speaking homes 
O Good — converse In English with little or no difficulty 



47, What are the major criteria used In the selection of pupils In grades prekinder- 
garten through 6 for participation In the ESEA Title Vn project In this School? 
- (Mark the major criteria) 



Pupils Whose ~ . Pupils Whose 

Primary or dominant Primary or Dominant 

Language Is Language Is Other 

English Than English 



a. 
b. 
c* 



e. 



f. 

g- 
h. 

1. 



Pupil surname 

Pupils* English-speaking ability 
Scores from.^esta to determine 
language ddmlnance of pupil 
Scores from standardized 

English — ^ , . . 



Scores from standardized 
achievement test written in 
tho primary or dominant 
language of the pupil 
Pupil grades . . ... 
Teacher recommendations 
Low income of family . 
dtlier (Specify): 




/ 
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48. Indicate'below which of the following services are part of the ESEA Title VII 
project for pupils in grades prekindergarten through, 6 in this school. (Mark 
all that apply) , , 

□ Academic Instruction lathe basic subject areas of mathematics or reading 

□ History and -cultural heritage associated with dominant language 

□ Non-English language arts 

□ Other academic subject areas 

□ Vocational subjects 

□ Guidance and counseling . ? 

□ Testing services 

□ Staff development ^ [ 

□ Community involvement 

□ Other (Specify): ; _^ ^ 



49^ Are the parents of children who participate in the ESEA Title VII project in- 
volved in any of the following activities related to the ESEA Title VII project? 



□ Volunteer tutor aide (unpaid) 

□ Community liaison , 

□ Field-trip aide 

□ Adult classes In English as a second language (ESL) 
Q Paraprofessional aide (paid) 

□ Planning, implementation, and evaluation of programs 

□ None of the above , * 



Note; Questions 50-62 ask about the handicapped pupils in this school. 'Handi- 
capped" refers- to pupils who have been classified as handicapped according 
to state guidelines. Where questions ask about types of handicapped pupils, 
report multthandicapped pupils according to their major handicapping con- 
dition. ' • 

50. During the 1974-75 school year, approximately how many handicapped pupils 
attend this school? (If none, write "0" and go to Section C, p. 22) 



(Mark all that apply) 




Number of handicappetl pupils 



17% 
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51. Does this school provide special educational or related services for these 
pupils? 

□ Yes 

□ No {Go to Section C, p. 22) 



52. Using the list below, indicate the types of ha4idieapped pupils you serve. (Mark 
all that apply for each grade distribution category) 



■ / 



Prekindergarten Kindergarten-6 Grades 7-12 



a« 


Trainable mentally 


1 — 1 
u 






b. 


Educable mentally 


□. 






c. 


Seriously emotionally 


□ 






d. 


Learning disabled . . . 


□ 


e. 




□ 


f. 




□ 


g. 


Hard of hearing .... 


□ 


h. 




□ 


i. 


Partially seeing .... 


□ 


j. 


Speech impaired .... 


□ 


k." 


Orthopedically 


□ 






1. 


dther health impaired . 


□ 


m. 


Other (Specify); 


. □ 





□ 

□ 
□ 
□ 

□ 

□ 

□ 
□ 



■ □ 

■ □ 

■ □ 

• □ 

- □ 

. n 

• □ 

■ □ 

• □ 

. □ 

• □ 

•. o 



/ , 
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What services for handicapped pupils are provided by this school? (Mark all 
that apply for each grade distribution category) 



d. 
' e, 

f, 

g- 
*h, 
i. 

J- 
k. 

- 1. 

m, 
n. 



p. 

q- 

r, 

s, 
t. 



Diagnostic assessment ^ * • • • 
Guidance and counaeling 
specifically for the 

handicapped ^ 

Parent guldanqe and. counseling • 
Emotional and social develop- 
ment skills 

Instruction in basic academic 

skiUs 

Tutoring . . . . . . . ... . . 

Language development skills . . 

Speech therapy . 

Auditory traifting 

Visual potor training 

Physical therapy 

Training in mobility skills . . . 
Training In self-car-a skills . . 
General health care ...... 

Special resources (e.g. , itinerant 

tutors, aides, « etc.) 

Resource room teachers .... 

^Learning centers 

Other special services ^.g. , pro- 
vision for flexible scheduling) . . 
pupil transportation ...... 

Other (Specify): 



Prekindergarten 



Kindergarten-6 



□ 



□ . A . . . 


I 1 

□ 


□ 


□ 


□ . . . . . 


□ 


□ . . r . . 


□ 


1 1 

□ . . . . . 




□ 


□ 


□ . . .; . , 


■ □ 


□ ...... 


□ 


□ . . . ;. . 


□ 


□ . . . J. . 


□ 


1 — 1 ■ 


1 — 1 

u 


□ ..... 


□ 


□ . . . . . 


□ 




■ □ 


□ 


□ 


□ 


□ 


□ 


□ 


□ . . . 






□ 
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54a. Do the handicapped pupils in graded' prekindergarten throngb 6 in this school 
have needs that are not being met at all at the present time? * 

□ Yes • * \ 

P No (Go to Q. 55) . , 

54b*. In your estimation, how many handicapped pupils in preklnderg?irten and kinder- 
garten through 6 are not receiving any special educational or related s'ervices, 
but should be? (If hone in a category, write "0"). 

„ Number of handicapped pupils in prekindergarten 

Number of handicapped pupils in kindergarten-6 



54c. Which of the special educational or related services indicated below are needed 
but are not being provided to handicapped pupils in this school? (^ark all that 
apply for each grade category distribution) \ 



a. 
b. 



c. 
d. 

e. 

f. 

g- 
h. 
1. 

J. 
k. 
1. 
m. 
n. 
o. 

p. 

q- 

r. 
t. 



Diagnostic assessment 

Guidance, and counseling 
epeciffcally for the 

handicapped 

Parent guidance and counseling . . 
Emotional and social develop- 
ment skills 

Instruction in basic academic 

skills 

Tutoring ' . v . . . . 

Language development skills . . . 
Speech therapy .......... 

Auditory training 

Visual motor training 

Physical therapy » . . 

Training iii mobility skills . . . . 
Training in self-care skills . . . . 

General health care 

Special resources (e.g., itinerant 

tutors, aides, etc.) 

Resource room teachers . . . . . 
Learning centers 

Other special services (e.g., pro- 
vision for flcrdblu scheduling) . . . 

Pupil transportation' 

Qtlio'r (Specify): , * 



Prekindergarten 

6 • • • 



□ 



Kindcrgarten-6 

. . . □ 



□ . . . 


. . . □ 


□ •. . . 


. . . □ 


□ . . . 


. . . □ 


□ . . . 


. . . □, 




. . . □ 


□ . . . 


. . . □ 


□ . . . 


. . . □ 


□ . . . 


. . . □ 


□ • . • 


. . . □ 




■ ■ . □ 


□ • • . 


. . . □ 


□ . • ■ 


• . . □ 


□ ■• • • 


. . .. □ 


□ • • • 


■ • • □ 


□ • • • 


... . Of 


□ • • • 


• • • □ 


n • • • 


• • . n 


□ • •. • 


. . . □ 



□ 
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55. Which of the following approaches are employed In this school to serve handl- 
capped pupils? (Mark the three major approaches) 



□ Malnstreamlng Into regular 'classrooms 
O Regular classes with special consultants 

□ Regular^^classes with itinerant teachers 

□ Resource roonj ' 

□ Part-time Special Education classes 

□ Full-time Special Educatioij cesses • 

□ Other (Specify): 




56. Which of the following special equipment or materials are available fo^use by 
handicapped pupils ip this school? (Mark all that apply) 

O Programmed learning rtiaterials - 

□ Audiovisual equipm^ or materials T 

□ Special equlprnq^t or materials for speech therc^py 

□ V Sensory aids 

□ Physical therapy equipment or supplies 

□ Mobility aids 

□ Special play equipment or materials 
> " □ Other (Specify): 



57. 



How many te^jichers (full- and part-time) of grades prekindergarten through 6 
in this school provide special educational or related services to handicapped 
pupils? (If none, write **0^^) 



Number of teachers of grades preklndergarten-6 



58. How many teachers*-ln this school pre certificated by this sta,te to teach handi- 
capped pupils? (If none^ write '*0'*) ' 

\ Number of teachers 
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59. How many teachers (full- and part-time) of grades prekindergarten throu^ 6 in 
this school provide special educational or related services to" handicapped pupils 
In the following? (Teacher§ may be counted more than once if they teach or pro- 
Vide services in more than one catiegory. If none in a category, write ''0.'^) 



Number of Teachers 

o£ Grades 
Prekindergarten-6 



...a, ^^"^legijlar classes • . \. \ 

. b. "^ISpet^ial classe%;for the handicapped 
c, .Individualized instruction / . . 



6Q. . Are |my of the following specialists on staff in this school to provide instruc- 
tional sugport^for haiidi^apped pupils? (Mark all that apply) ) 

I I Speech therapists n 

O Psychologists 

O Educational diagnosticiSlis 

□ Social workers 
' Physical therlpists 

r~] Occupational'therapists " 

□ Other (Specify): • 



61. Is this school physically designed to accommodate cliil^ren with orttiopedic 
handicaps (i.e. , washroom facilities, drinking fountains, ramps)? 



□ Yes 
O No 



-4- 



62. What are t^e major criteria used in determining th^eligibility of eljfementary- 
levelij^^cipauts in specif educational or related services for handicapped 
pupils? (Mark tiie three major critajria) 
* - f 

□ Standardized test scores l 

□ Grades v \ ^ 

□ plecommendations of teachers or professional staff members 
*" CD Parental requests _ ^ " - \^ 

n Agency referrals . \ . [ 

□ Type of handicap 

□ ^ Severity of handicap 
^ □ Medical referrals V ' ^ 

□ OlW (Specify): ' : 



\ 



EDUCATIONAL BROADCASTING' 



63. What is t^^ total number of teaching stations In this school and how many of 
these are being used for Instructional purposes in this school year? 

Total number of teaching stations 

1^ ^ Number of teaching stations in use 



64. In column A , indicate the total number of televisions and videotape recorders 
in this school that are owned by the school (or the district) for instructional 
purposes. (If none in a category, write "0'*) 

In column B . enter the i^^ber of these that are in working condition; i.e. , the 
quality of picture and s<^imd are suitable for instruction. (If none in a category 
write »»0") / * ^ 

k \ ' B ' . 

TBtal Number N^^er in ^ 
^ Working Condition 



a. Televisions , . , . 

b. Videotape recorders 



65* Using your best professional judgment, rate the closed circuit television 
^eygtem and programming for your school as of January 31, 1975. (If these 
are not available to this school, mark that a^ltemative. Mark only one choice 
. in each, horizontal row. ) < 

Available ^^^^^^^^ tiood Fair Poor 

a. Adequacy (suitabili]ty) of closed 

circuit television system for ^ ^ 

instruction □. Q. □ 

b. Quality of closed circuit tele- 
vision programming for 
instruction 
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66. For which subject or activity areas does this school use television programming 
for In-school instruction for pupil»? (Mark all that apply) 

Q None. ^ 

□ - Remedial reading 
- □ Other reading ^ ' * 

, Q Other English language arts ' 

□ Languages other than English v • 

□ Mathematics 
n Science 

Social studies " 

□ Environmental quality and poUutioa " ' 
O Occupational familiarization 

□ ^Music/art * . » 
" □ Other (Specify): _^ 




COMMENTS 

You have completed this questionnaire. However, if you have any comments-ybu 
wiuld like to make regarding the clarity of the .questions, definitions, andinstruc- 
tiongfcUsed in this questionnaire, please use the rest of this page, THANK YOU 
FOR YOUR HELP. • ^ ^ > 
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PERSONAL CHARACTERISTICS 



1 • What is your highest level of academic achieve- 
ment? (Mark one) 



O 
O 

o 
o 
o 
o 
o 
o 



A doctord I degree 

Specialist degree (MA+30) 

A master's 'degree or equivalent 

A bachelor^s degree 

An associate's degree 

No degree, but some college courses 

A high school diploma 

Other (Specify): ' 



2, Do you hold a state teaching certificate? 

O Yes 
O No 



3. What is your sex? 

OMalc 
O Female 

4. How would you describe yourself? (Mark one) 

O American Indian ■ 

O Black/Negro 

QOrientat 

O Hlspan?c 

QWhUe/CauCasian 

O Other (Specify): 



6. How old are you? (Mark one) 



O Under 21 
0.21-25 
026-30 
031-35 
O 36-40 
041-45 . 



O 46-50 
051-55 
056-60 
061-65 ^ 
O 66 "bT older 



How many years, including this year, have you taught in 
any gra^e prekfndergarten through 12? Include both 
public and private school experience. (Mark one) 

O Less than one year 
©At least I year, but less than 3 years 
©At least 3 years, but less than 6 years 
©At least 6' years, but less than 10 years 
©At least 10 years/'but less than 20 years 
^ 20 ye|rs or more | ' - ' , 



7, What is your current annual salary from this 
school district? 



10. 



11 
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8, Are you currently a full-^time employee of 
this school district? 

O Yes 
O No 



9. How many years, including this year, have 
you worked at this school? (Mark one) 

O Less than one year 
© At least 1 year, but less than 3 years 
® At least 3 years, but less than 6 years 
©At least 6 years, but less than 10 years 
© At least 10 years, but less than 20 years 
• © 20 years or more 



Since June 1974, have you received any 
preservice training provided by the school 
district or school? 

O Yes 

O No (Go to 13) 



How many clock hoyrs of pfe^ervice training 
have you received since June 1974? "^If your 
preservice training included coTlege courses, 
be sure to indicate clock hours — not semester 
or quarter hours. (Mdrk one) 

O Less than 5 Hours 
06-10 
011-15 
©16-20 
'©21-26 
©26-30 
©31-35 
©36-40 

© More than 40 hours 



18-1 



12. In which of fhe following- areas have you 
received preservi^e training since June 
1974? (Limit your responses to the most 
important <ireas) t 

y 

O General orientation and administrative information 

O Academic instruction in reading 

Q Academic instruction in other subjects 

O Tridividualized instruction 

QTeam teaching 

O Identification of pupil problems 

O Treatment of pupil problems 

O Evaluation methods 

O Dissemination strategies 

O Bilingual/bicultural education 

O Education of handicapped pupils 

O Other areas (Specify): 



13, Have you received any inservice training 
'from- the school district or school since 
June 1974? 

O Yes' 

b No (Go to Q. 17) 



14. «How many j:\6ck hours of inservice traiq|ng 
have you received. sinpe June 1974? If your 
inservice troining included college courses, ^ 
be sure to indicate^ clock hours — not semester 
or quarter hours. (Mark one) 



O Less than 5 hours 
06-10 
011-15 
016-20 . 
021-25 



O 26-30 
03-1-3S 
O '38-40 

O More^than 40 hours 



15. In which of the following areas have you 

received inservice training since June 1974? 
(Limit your responses to th^ most important 
areas) 

O College courses in using television or 
radio in instruction 

O Acadexnic iostructian in reading . 

O Academic instruction in other subjects 
O Individualized Instruction 
O Team teaching 

O Identification of pupil problems 

O Treatment of pupil problems 

O Evaluation methods i 

O Dissemination strategies 

O Silipflual/blcultural education 

O Education for handicapped pupils 

O General administrative procedures and 

.recordkeeping . f ^ . .-: 

O Other emphasis (Specify): 



16. For which of the following target groups 
Was the inservice traini>ng in whfch you 
participated specifically desig'ned? {Mark 
all that apply) 

O No specific target group 

O Educationatty deprived pupils 

O Academically gifted pupils 

O Migrant pupils 

O Handicapped pupils 

O Neglected or delinquent pupils 

O Pupils from low-income families 

O Pupils from homes where the primary or dominant 

language is other than English ^ - ^ 
O Other target group (Specify): " *_ 
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17. Which of the sources listed below provide 
funds for any projects in which you teach? 
(Please check with your principal if you 
don't know the answer to this question. 
Mqrkall that apply.) 

/r 

O ESEA Title I (children from low-income 
areas, neglected or delinquent 
children) 
O ESEA Title I Migrant 
O ESEA Title III, Section 306 
O ESEA Title VII (bilingual) 
. O Education for the Handicapped Act> 
Part B (formerly ESEA Title VI) 
O Other programs for the handicapped 
O CommunJcations Act of 1934, Title III, 
Part IV (Grants for Noncommercial 

Educational Broadcasting Facilities; 

Corporation for Public Broadcasting) 



18. During the course of the school year, yon 
may have had to pay extra attefntion to some 
aspects of teaching. Please look at the list ^ 
below and indicate which items are of special 
concern to you. (Mark all that apply) 

O Many pupils enter below grade level and 
need special or remedial help 

O The range of ability among pjypils is so 
large that it is difficult to prepare 
lessons that are suitable for the whole 
class 

O Many pupllslhave special or individual 
learning difficulties requiring special 
curricula or special classes 

O Many pupils are so able that they are 
bored by the standard class material 
or pace 

O Many pupils have behavioral problems 
' that result in a great deal of learning 
time being spent on discipline 




During an average school day, how much 
Ime do you have to yourself to prepare 
essons or work with individual pupils 
'(excluding lunch period)? (McJrk one) ^ 

O None 

O 15 minutes or less 

O Between 15 nrfinutes and an hour 

O An hour or more 



B 

CLASS CHARACTERISTICS 



Note: For all the remaining questions in this 
questionnaire, the term "this cjass" 
refers to the class for which you are 
also completing Pupil Questionnaires." 



20. Indicate below the grade or type of home- ^/ 
room class for which you are reporting . 
(Mark one) 

O Preklndergarten 

O Kindergarten 

O Grade 1 

O Grade 2 . 

O Grade 3 " ' 

O Grade 4 

O Grade 5 

O Grade 6 " . ^ 

O Ungraded class for handicapped pupils 

O Ungraded class (other than for handicapped 

pupils) 
O 'Multigraded class 



21 . How many pupils are enrolled Tn this class? 

Number of pupils 
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22, What language — other than English — do you 
use to Instruct this class? Do not Include any 
Instruction you might give when teaching a 
foreign language. (Mark one) 

O None (English is the only language used) 

O An American Indian language 

O An Oriental language 

O Spanish 

D Other language (Specify): 

23, In what subject areas do you use the language 
that you Indicatetf Tn question 22? (Mark 
all^that apply)^ 

O Reading 

O Mathematics 

O English language arts (excluding reading) 

O Other (Specify): ' 



24, How many of the pupils in this class are 
members of the following groups? (If none 
In a*category^ Wltte "Q") 



Numbf o> Puplli . 



L Amorican Indian 

b. Black7Negro . . . 

c. Orlortto! ...... 

d. 'Hfipanfc 

e. White/Caucasian. 

f. Other (SROCjlvlt 



25. Approxifpately how many of the pupils in this 
vclass are from homes where the primary or 
>^ dominant language is other than English?" ' - 

* -Number of pupils 



26. In an average week^ are most of the below- 
average readers in tl^Js class assigned any of 
the following? (Martc all that, apply) 



Additional reading time in class (while 
other pupils are engaged in other 
activities) ^ 
Special reading classes Qutside the regular 

class (within school hours) 
After-school tutoring 
Supplementary homework 
Other (Specify): 



O 

o 
o 
o 



None of the abdve 



27. In an average week, are most/of.the below- 
average pupils in math assigned any of the 
following? (Mark all that/apply) 

O Additlonat math time In doss (while other 

' pupils are engaged In other activities) 
O Special math classes outside the regular 
class (within school hours) 

O After-schooh tutoring—^ — ^ , 

* O Supplementary hqmework 
' O Other (Specify): ^ / 

O None of the above 



,28. Approximately how many of the pupils In this 
class do you expect will be able to wpjk qt 
'^rad^^ifive I or above next year? 

Number of pupils 



29, In your judgment^^fiow many of the pupils 
In this class are likely to complete: 



Number of Puplli 



a. E^omentary school . . . 

b. school 

Some college 

. , • f 
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SPECIAL PROGRAM PARTICIPATION 



Note: Pteaje^heckwrth your principal If you 
don't know the answers to any of the 
questions In this section • 

The term "this class" jgf^rs fro'1=hS'"6.lgss 
fot which you are alsTocompleting Pijpil 
Questionnaires. 



32. 



30. Do any pupils in this class participate In 
projects funded by ESEA Title I? (Do not 
Include ESEA Title I Mlgfont herejf 



O Yes 

O No {Qfx^^tQ Q. 34) 

31. How is instructio^ovided In the ESEA 
Title I projects? (Mark one. Do not 
Include ESEA Title I Miggonthere.) 



O Pupils receive Instruction all day in the 
regOlar classroom as part of an ESEA 
Title I project 

O Pupils receive instruc'tfon part o Fthe 
day In the regular classroom as part 
of an ESEA Title I project 

O Pupils go outside of the regular class- 
room for part of the day to receive 
.Instruction as part of on ESEA Titled-- 
project - - 



How many pupils. In this. class partlclpaTe' 
In projects funded by ^EA Title I? (Do 
not Include ESE^Title I Migrant here) 




Number of pupils 



33. How mqny pupils in this class participate in ' 
ESEA Title I projects in the following subject 
areas? (Pupils may be counted morS than once 
if they participate In more than lone subject 
area'. If none In a category, write "0." Do 
not Include ESEA Title I Migrarlt here .) 



I^umber of Pbpllg 



a. Keadir^^,^ • 

b. Mathematics 

c. English language arts 

* (excluding rea<JIng). . . . 

d. Bilingual instruction... 

e. Er^glish as a second 
language 

f. «-*Other (Specify): 



c 



34. 



Do any pupils In this class participate In 
projects funded by ESEA Title f Migrant? 

O Yes 
^ No (Go to Q. 38) 



35. How is instruction provided in the ESEA 
Title I Migrant projects^ (Mark one) 



Pupils receive Instruction oM day In the 
regular classroom as part of an E5EA 
Title I Migrant project 

Pupils receive Instruction part ^i^edo^, 
Jn tlfe regular classrobm as part of an ^ 
ESEA Title I Migrant project 

Pupils go outside of the regular classroom 
for part of the day to receive Instruction 
as part of an ESEA Title I Migrant project 




O 



Z6p How many pupils In this class participate Ih 
projects funded by ESEA Title I Migrant? 

V Number of pupils-. 
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37. How many pupils in fhis class participate in 
ESEA Title I Migrant projects in the following- 
subject areas? (Pupil? may be counted more 
than once if they participate in mor^ than one 
subject area. If none in a category, write 
"0") 



Q. Reading 

b« Mathematics. 



Numbejr of Pupils 



c« English language arts 
(excluding reading)., 

d. Bilingual instruct 

e. English as a second 
language , 

f. Other (Specify): 




38. Do any pupils In this class participate in proj- 
^ ects funded by ESEA TIHe III, Section 306? 

O Yes 

O No (Go to Q. 40) 

o 

39. How many pupils jn thTs class participate in 

' projects funded by ESEA Title III, Section 306? 

[ Number of pupils 



40. Do any pupils In this qIoss participate in 
projects funded by ESEA Title VII? 

O Yes " 

O No (Go to Note before Q . 45) 



41. How many pupils In this class participate In 
projects funded by ESEA Title VII? 



Numbfer of pupils 



42. Approximately what percent of the time is 
the instruction In this class provided In the 
language of the ESEA Titfe VII project in 
this school? (Mark one) 

O None 

O 1-25% 

O 26-50% 

O 51-75% 

O More than 75% 



43. Do you have a teacher, aide who assists you 
in the classroom? 

O Yes 

O No (Go to Note before Q . 45) 



44. What percent of the teacher aiders time Is 
. spent in the following activities in the class- 
room ? (If no time is spent in an activity]! 
enter a zero. The total should equal 100%.) 

Percent of Time 



a. 

b. 
c. 
d. 
e. 
f. 

9- 
h. 



Clerical or administrative 

duties 

Custodial duties 

Preparation of materials .. 

Class instruction 

Test administration 

Reinforcement activities 

with small groups 

Individual pupil attention. 
Other (Specify): 



100% 



Note: 



Questions 45)^-54 ask about your experience 
teaching har^dlcapped pupils. "Handicapped*' 
refers to pupills who have been classified as 
handicapped according to state guidelines. 



/ 
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4. . 



45. Are there any handicapped pupils in this 
class? " • 

O Yes 

Q No (Go to'Q. 48) 



46. How many hondicqpped pupils are in this 
class? 



47. 



NMjnnif>er of hondicapped 
pupils 

9 



How have these pupili been classified ' 
according to state guidelines? (Pleose 
.check with your principol if you don't 
know the answer to this question. Mark 
all that apply.) 

O Trainable mehtolly retarded 

O Educable menta I ly retorded i 

O Seriously emotionally disturbed 

O Learning disabled 

O Deaf-blind 

O Deaf • 

O Hard of hearing 

O Blind 

O Partially seeing . 

O Speec'h Impaired 

O Qfriiopedically har^dicapped 

O -Otfi^r health impaiye<J 

O Other (SpecYfy): 



48. Do you teach ony haridlcaf^ped pupils* In 
this school? H . . 

O Yes 

O No (Go to Section D) 



49. How many hapdicapped pupils do you teaph 
}n this school? : , 

> ' Number oP handicapped 

pupils 



50. Are you certificated ^y thl^ state to teach 

capped pupils? 



or provide services f^j 



O Yes 
O No 



l^jj^di 



51 • How mony years have you taught or provided 
services for hondicopped pupils? (Mark one) 

O Less thon 1 yeajX 

O l-2yeors ^ / 

0 3—5 years 

O 6— lOyeors 

O 11— 20 years 

O Mbre than 20 years ^ 



52. In the last five years, have you received any/ 
speciol troining that prepared you for providing 
special educational or related services to 
handicop^ed pupils? 



O Yes 

O No (Go to Section D)^ 



53, 



What kind of special troining have y^u received 
to date during this schpol year (1974-75) that^ 
prepared you' for providing special educational 
or related services to hondicapped pupils? 
(Mark all that apply) 



O 
O 
O 
O 
O 



None (Go to Section D) 
Undergraduate Special Education courses 
Postgraduate Special Education courses 
Workshops/lnstitutes/inservice training 
Other (Specify): 



54. 



How many clock hours of special training have 
you recslved to doteauring this school year 
(1974-75)? If this special training included 
college courses, be sure to indicate clock 
hours— not semester or puarter hours^ 

llock hours 
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Note; 



EDUCATIONAL BROADCASTING 



The term "this class" refers to the class 
for whfch you are also completing Pupil 
Questidnnaires. 



55. Is a ^elevtslon set (in good working condition^ 
with quality of picture and souijid suitable 
for Instruction) re|adily available to pupils 
In thljS class for instructional purposes? 

>0 Yes 

O No (Go to Q . 59) / 



56* Approximately how many hours a* week during 
the 19?4-75 school year is a television set 
(In good working condition, with quality of 
picture and sound sqitabFe for instruction) 
readily available to pupils in this class for 
Instructional purposes? (Mark one) 







^020 


O30 


O 40 hours ~ 


loss 


On 


021 


031 


or more 


02 


* Ol2 


022 


032* 




03 


Ol3 


023 


033 




04 


Ol4 


024 


034 




05 


Ol5 


025 


035 




06 


Ol6 


026 


036^^ 




07 


Ol7 


027^ 


037 


K 


08 


Ol8 


028 


038 




09 


Ol9 


029 ' 


,039 


* 



57. Approximately how many hours a week durhg 
the 1974-75 school yepr does this class use 
television In. school for Instructional purposes? 
'(Mark one) 



O None (Go to Q J 



O 1 hour or 

02 
03 * 
04 

Ob 



Oe 

07 
08 
09 

Oio 
Oil 



■^0^2 

Oi3 

Ol4 
Oj6 
Ol6 
Ol7 



58. In which of the subject areas below does this 
class use television for in -school instruction? 
(Mark all thgt apply) 

OArt 

O Remodial reading 
O Other reading 

O English language arts (excluding reading) 

O Languages-other than English 

O Health and safety in daily living, physical education. 

and recreation 
O Home ecQnomics * 
O Industrial arts 
O Mathematics 
O Music 

O Natural sciences - < 

O Social sciences/social studies 

O Special Education 

O Environmental quality and pollution 

O Public affairs 

(J Occupational familiarization 

0 Other (Specify): - 



59. During the 1974-75 school yebr, approximately 
^ what percent of this class uses the television 
series Villa Alegre for in-school viewing? 
(Mark one) • 




/ O None 
' O 1-25% 
26-50% 
51-75% 
More than 75% 



60. During the 1974-75 school year, approximately 
what perqeot bf this class uses the t(&levision 
series The Electric Company for in-school 
viewing? (Mark one) ^ 



fGo toQ. 68, pdge 10) 
1-25% r- 
26-50% ; = . * 

51-75% ^ * 

More than 75% " 
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61 . Approximately how many times a week 
during the 1974-75 school year does this^ 
class use the television series The Electric 
Company for tn-school viewing? (Mark one) 



ONone 
OOnce a week 
or less 

02 

03 
04 

Ob 



06 
07 

Ob 

09 

O 10 times a 
week or 
more 



/ 



/ 



62. 



For which types of pupHs if/this class do you 
feel The Electric Compan^ M most useful? • 
(Mark one) / , • ' 



63. 



O f>u^ll$ with below average ability 
© Pupils with average^blllty 
<® P^upils with above^^verage ability 
© Equally useful for oil types of pupils 
© Not very usefi^t for any typo of pupil 



To what extent has the use of 'The Electric 
Company television series madelTiis class 
use other television programming more 
during- this school year (1974-75) than in 
past School years? (Mark one) 
* 

©A great d^al 

§, Somewhat 
Very little 
0 Not at all 
© I don't know 



64,; In what setting do most of the pupils in this 
class view The Electric Cornpany television 
' series at scKool? (Mark all that apply) 



O 
O 

o 



ERIC 



Individually 

Some of the pupils vl^w it Ixi^mall groups 
The class views it as a unit (for example, 

self-contained classroom) ^ 
By "doubling up" with ^ther classes in 
. the Waching stotion with- the television 

set , 
In qn auditorigm 01^, other large room 

Other (Specify); - - ^ ' 



65. When viewing The Electric Company ^ are the 
pupils in this class free to decide where they 
will sit? , 

O'Yes 
O No 

66. Are any the following actlvitHes performed' 
by the pujDils in this clqss as part.of viewing 
The Eleptric Company? (Mark ^^Yes" or "No" 
loTeach) " 
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Ym Nor 
.0 .0 



I, Advance preparatlort . - 

(for example, the pupHs are given a 

special introduction to the program 

or a review of the program guide) .... 
). Active participatiofT during the program 

(for example, pupils sounding out words 
•*or letters, singing along with the songs. 

«tkin(f/answering questions while the 

program is being viewed, etc?) . . . . O . .O 

Follow-up after the program * • 

(for example, reviewing words or concepts 

presented in the program, asking/answering 

queitions afi^Jt the program is over, etc.) . 

ft. ■ \ ■* 



.0.0 



1 



00 



67, During the 1975-75 school year, how much 
have the pupils in this class learned about 
each. of the skills listed below as a result of 
viewing The Electric Company te levision 
series? (For each skill, mark>he one box 
that best describes how much your pupils 
have learned and then go to COMMENTS, 
page 11) ^ 



a. Decoding skills 
(i.e., pbonetic skills: 
stritegi^s for trans- 
lating graphic symbols - 
into speech Sounds) 

b. Punctuation 

(i.e., understanding 
the function of 
common punctua- 
tion symbols) 

a Reading comprehension 
(i.e., extracting meaning 
from sentences or ex- 
tended passages) 

d. Sight words 

(i.e., ph(Kietically irregular 
words taught as w^ole 
yvords) 

0. Using contexts 

(i.e., utilizing information 
from the rest of the sen- 
tence to figure out Gn- 
^ familiar words) 



'Pupils have learned . . . 

t 3 3 4 5 

A I 
QTMt Somt- Vfry Not don't 
dul what littit It 111 Know 

.o..o..o..o.d 



0..0..0..0 o 



0..0..0..0..0 



O. 0..0..0 .O' 
.0..0...0. 0..0, 



(Go to/COMMENTS, page 11) 



iciionS. 
/isor) irivi 



68. Whith of the following help to explain why 
this cla^s is not using the tQlevision series 
The Electric Company during the 1974-*75 
scRool year for rn-school viewing? (Mark 
all thaKapply) 

O There is no television avaijpble for use by. 
this class. 

'O* The program is not reino shown on a tele-^ 
, vision channel that cqn be received in 
this class.' 

O The television received avdilqble to this 
qlass is VHF (channels 2-13\and thtf 

^Q** T^^'Q Electric Company is UHF 
^channds 14-83^! 

O' The p0rson responsible for instruci 

(principal, teacher, or supervisor) irivtWs 
class does not feel that te levision 'ns a very 
useful medium f^r instruction in readthg 
or-othefj instruction areas. 

O The persor> responsible for instruction in this 
class considers the content of The Electric 
Company to be inappropriate for the . 
instructional needs of the pupils. 
» O The pupils in this class are viewing The 
Electric Company at home; it^s not 
" necessary to use it in school • 
/ O The person responsible for instruction in this 
class prefers to have the pupils view a 

^ television program of local origin rather 
thdn one that is produced elsewhere. 

O It doesn't fit in with the regular reading 

instruction program in this class. 
•O The' Electric , Company program is not in "good 
taste"; i.e., the "tone** and "style", of . 
the program is unacceptable to the person 
responsible for instruction in this class. 

O The person responsible for instruction in this 
class lacks sufficient experience to t)se a 
television approach for the teaching of 
reading . 

O The' program is not on at a suitable time of 
the day . 

O The person responsible for instruction in this 
class doesn't know enough about The 
Electric Company series to arrive at a , 
decision regarding the desirability of using 
it in a reading instruction program for the 
pupils in this class. 

O Other (Spe<^ify): " - 
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dpMMENTS 

^ / •' 



You have^om^j^ted this questionnaire. However/ 
if you have qny comments you would like to make 
regarding 'the clarity of the questions, definitions, 
and ifi'structions used in this questionnaire, please 
uso/he rest of this page. THANK YOU FOR 

YOUR help: 




\ 



/ 



GENERAL CHARAa ERISTICS 



1« In what grado or type of homeroom it this pupit enrolled 
now? (Mark, one) 

O Prekindergarten ® Grade 3 

■© Kindergarten ® Grade 4 

.© Grade 1 ©Grade 5 

© Grade 2 ® Grade 6^ 

© Ungraded class for 
handicapped pupils 
® Ungraded class (other 
than for handicapped 
puplts) 
®Multigraded class ' 

2. If this pupil is In an upgraded or a multigraded 
clasS/ what is his equivalent grade level? If 
this pupil is handicfi'pped, answer in terms of 
reading level . (Mark one) 

O This pupil is not in an ungraded 
or iTiultigradod class 
O Pr^jjpdergarten ; O Grade 3 
O KxMj^^an^n / O Grade 4 

O Grade^ 1 ©Grade 5 i 

©Grade 2 ©Grade 6 



When did this pU^il first enroll In this school? (Mark one) 

© At the begin/iing of this school year 
© After if)o begin ning( of this school year 
® Last year 
©2 years (rgo ^ 

© 3 yoari ago , - ^ 

© 4 years ago or tporo 
© I don't know 



Not counting this school, how many elementary schools 
has this pupil attended since beginning first grade? 
(Mark one) 



How friany days was*thii pupil absent from school between 
the beginning of the current school year and January 31, 
l975?(Mark one) 



O No absences 
© 1 -5 days 
©6-10 daVs 
O 11-15 days 



© 16-20 days 
©21-30 days 
© More than 30 days 



7. 



8, 



In your judgment, how likely is this pupil to complete: 
(Mark one for each horizontal row) ^ 

Vary likily Not vtv Hkity 

a. Elementary school © ® 

b. High school © © 

c. Some college © © 



In your judgment, how does this f)upirs general acadamic 
performance compare with others in his class? (Mark oh^ 

© His academic performance is consistently better than the 

average for his class \ N 

© His academic performance is about the same as the 
average for his class 

© His academic performance is below the average for 
his class 

Considor this pupil's general academic performance at three 
poinU in time during this school year: (a) when the Pupit 
entered this class, (b) now, and (c) as anticipated at the 
end of the school year. (Mark one in each vertical column) 



9. 



O 
O 
O 

o 



None 
1 
2 
3 



O 4-5 

O 6-t10 

O More thpn 10 

O Idon'ricnow 
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£ntf ing 



Below grade level © . 

At grado level © . 

Above grado level © . 



Now 

.©. 
.©. 
.©. 



c. 

End o« Ywr 



o 
© 

.© 



Does this Pupil receive the majority of his instruction in 
any of the following areoi from you? (Mark all that apply) 

©Reading 
© Mathematics 

©English language arts (excludfng reading) 
OOthcrja^dcmic subtoct areas 
©Nbrto of these 



/lO« Gin thts pupil be classified as any of the 
following (see definition 11)? (Mark all 
thatappi/) 

O Educationally deprived 
O Academically gifted 
O Migrant pupil 
O Handicapped 
^ Neglected or delinquent 
O Child from low-income family 
O Child from home where primary- or 
dominant language is other than 
English 
O None of these 



11. What is this pupil's sex? 
O Male 

O Female ' 



12. How would you characterizf thii pupil? (Mark one) 

O Amtrican Indian 
® Black/Negro 
©Oriental 

© Hlipanic . ^ 

© Whita/Ciucisian 

©Other (Spoclfy): ^ 



■1^ 

13. Where was this pypil bom? (Mark one) 

O The United States (50 states plus D.C.) 

O Puerto Ric9 

O Canada ^ 

O An Oriental Country 

O Cuba 

O Moxtco' 

O Centra I AmeHcah Country 

O South American Country 

O Other Country "(Ipecify): 

O I don't know 



Hpw long hat thit pupil lived Ih the Unitiid States? 
(Mark one) 

O Since birth O 3-5 years 

O Over \ 1 years O 1 -2 years 

O 9-11 years O Less than 1 year 

O 6-8 years O I don't know 
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15. 

- / 



16. 



How long has this pupil lived in this area? (Mark one) 

O Since birth O 3-5 years 

O Over 1 1 years O 1 -2 years' 

O 9-1 1 years O Less than one year 

O 6-8 years O I don't know 



Has this pupil's father, mother, or guardian had a career 
In tha military or foreign service with the U.S. government? 

©Yes 
© No 

© I don't know 



17. What is the primary ^r dominant language 
spoken in thts pupilTTiome? (Mark one) 

b English 

O An American Indian language 

O An Oriental language 

O Spanish 

O Other (Specify): 

O 1 don't know 



18. This pupil's ability to converse In English Is: (Mark one) 

© Very limited - he conducts little or no conversation 
in English 

® Limited - he converses in English, but with more 
difficulty than would be expected of a child from 
an English-speaking home 

0 Good - he converses in English with little or no 
difficulty 



19* Counting this pupil, hov/ many people are currently living 
in this ptipil'i ^household? (Mark one) , 

02 08 

03 07 

O 4 O 8 or more ^ 

O & ©I don't know 

20. Would you classify this pupil as one coming from a 
low-income family? 

©Yes 
©No 

21, What is your definition of low*income? 
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IS? 



22* In your opinion, how educitlon«lty tupportlve It this . 
pupll'i homB.environmint?(Mafl< one) 

0 Very supportive 

® Moderately supportive 
. ® Not very supportive 
^ © I don't know 

23 • Does this pupil have persistent academic problems that 
require special assistance or placement into a special 
program to address his specific needs? 

OYes V 

O No (Go to Q. 25)- 



'24. In what subjecf areas does this pupil have 
these problems? (Mark all fhat apply) 

O Reading 

O Mathematics^ 

O English language arts (excluding reading) 

O Other academic subject area (Specify): 



25. 



26, 



O 
O 
O 
O 
O 

ERIC 



U thfs pupil performing one or more years 
below grade level (as measured by standard- 
ized achievement tests) in any of the following 
subject areas? (Mark "Yes" or "No" for each) 

If this pupil is in prekindergarten or kinder- 
garten, mark here O and go to Q. 26. 

Don't 

Y»i Nq know 

Reading O..©..® 

Mathematics O..®..© 

Enqilsh longuogo arts 

(excluding reading) 0..®..® 

Other academic subject area (Specify): 

' 0..0..0 



Does this pupil have iny persistent problems that require 
profetsional Mtistanco or other typoi of special assistance? 
(Mork oii ihor oppi/) 



This pupil does not have any persistent 

problems 
Physical or sensory handicap 
Social or emotional problem 
Mental handicap s 
Medical or dental problem 
Other (Specify): 



^ 27, In your judgment, can this pupil be de- 
fined as educationally deprived? 

" O Yes 
O No 



Notfe: 
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FEDERAL PROGRAM PARTICIPATION 



Please check with your principal if you 
don't know the answer^ to any of the 
questions in this sectfon. 



28. boes this pupil participate In any projects 

funded by ESEA Title I? (Do not IncludeiESEA 
\\\\o I Migrant here) { 

Yes 

No (Go to Q. 30) 



29, what are the subject or activity areas of the 
services provided to this pupil through ESEA 
Tlile I? (Do not include ESEA Title I Migrant 
he|;e* Mark all that apply.) 



odlns Irufrucrlon 
fhomorfcs Insfrucrlon 
ipilsH longuogo ortt Insfrucflon 
(excluding reodlng) 
Bilingual Intfrucflon 

llih as o second language 
0(lKor ocodemlc Inifrucflon 

(Specify): 

G ildanco ond counseling lorvices 




O 
O 

o 



o 
o 



Alkndanco ond loclal work lorvlces 
Health lorvlcor (Including medlcol ond donfal 
•xomlnorloni, Imlrucflon In Health and Iwalrti 
cart) 

Food torvlces ^ \ 

Or lor sorvlces (Specify)! ^ 



30. Does 
by 



this pupil parrlclpafe In on/ pro|ecrf funded 
Tirle I MIgronf? 



ESElA 



O No'(Go(oQ. 32) 



31 • What are the subject or activity areas of the 
services provided to this pupil through ESEA 
Title I Migrant? (Mark all that apply) 

O Reading instruction 

O Mathematics instruction 

O* English language arts instruction 

(excluding reading) 
O' BfJingual instruction 
O English as a second language 
^ O Other academic instmction 

(Specify): 

Occupational skills familiarization 



o 
o 
o 
o 



o 
o 
o 



Guidance and counseling services 
Attendance and social work services 
Health services (including medical and 

dental examinations/ ihstruction In 

health and health care) 
Food services 
Clothing 

Other services (Specify): « 



32. Does this pupil participate in any projects 
funded by ESEA Title III, Section 306? 

O Yes 

O No (Go to Q. 35) 



33. What are^ the subject or activity areas of the 
services provided to this pupil through ESEA 
Title III, Section 306, funds? (Mark all 
--that apply) 

O Rcoding instruction 
OMathomotics ihstruction 

O English tanguago arts ihsuuction (excluding reading) 
QOther acadtimic instruction 
(Specify): 

'•^s """" 

0<3i^ance and counidlina sor vices 
QAtfondanco and social wpi^k services 
O Health services (including mectigal and dental 
examinations, instruction in health and 
health care) 
Qpood services *' ' ^ 

O Other scfVicosf (Specify)^' 





34* On what boilt wos thh pupil lelecfed to partlclpot* In 
fhm ESEA lUlm 111, Stctlmi 306, protect? (Mark ono) 

0 Basis of selection is unknov^ 
© No special criterion employed 
0 Standardized achievement test scores 
0 Other standardized test scores 
© Pupil grades 

© Special fieeds of pupils (e.g.« handicaps, problems 

In disl^iplino, attitude) 
0 Teacher recommendations based on educational 

needs 

0 Teacher recommendations iMSod on other needs 
0 Low Income of fomlly 
0 Parent or pupil request 

©Other (Specify): 

35. Does this pupil participate in any projects funded by 
ESEA Title VIl? 

O Yes 

O No (Go to Section C) 



36; 



37. 



On what basis was this pupil selected to 
participate in the ESEA Title VIl project? 
(Mark all that apply) 



O Pupn*i lurnomo 

O Puptl"s Engllth-fpooking oblUfy 

O Scores from tests to determine longuogo 

dominance of pupil 
O Scores From stondordlzed ochlevement test 

written In EoQlIsh 
O Scores from stondordlzed ochlvement test 

written In the primory or domlnont longuogo 

of the pupil 
O Pupil grddes 
O Teocher recommenddtlons 
O Low fncome of fomlly 

O Other (Specify): 



Indicate below the services provided to this 
pupil as part of the ESEA Title VI | pro|ect? 
(Mark all that apply) 

O Acodemic Instruction In the bosic subject 

orecs of mothemotlcs or reading 
O HIttc/ry ond culturol herltoge osiocloted 

yvtth domlnont longucge 
O Nc^h'-Engllsli longuoge orrs 
O Olhor ocodemfc fub|ocf oreos 
O Vocotlonal subjects 
O Guldonce ond counseling 
O Testing services 
O' Stoff dovelopmont 
O G)mmunlty Involvement 
O Other (Spec I f/)i 
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PROGRAMS FOR THE HANDICAPPED 



Note: Questions 38-47 ask about this pupil's 
participatlon\ in programs for the handi- 
capped. "Handicapped" refers to pupils 
who have been classified as handicapped 
according to state guidelines* 

Please check with your principal if you 
don't know the answers to any of the • 
questions in thfs section* 

38* Is this pupil classifie^as handicapped? 

O Yes / 

O No (Go to COMMENTS, page 6) 



39, Please specify how this pupil is handicapped 
(see doftnit}or^4). (?ase your answer on state 
guidelines. Please cfiock witljyour principal 
If you don't know the answer tolHs^uestion . 
Mark all that apply*) 

O'i'rolnablo montally rotordod 
O^^ucabto montally rotardod 
"OSorioutly omotionolly disturbed 
O Learning disabled 
O Doaf -blind 
ODeaf 

OHard of hoaring 
O Blind 

OPof^^olly loolng 
OSp^iQch impaired 
OOrthopodically handicapped 
O Other health impaired 

O Other (Specify): ' 

O • don't know 



40. Did this pupil receive any prekindergarfen 
(early childhood) education? 



O 
O 

o 



Yes 
No 

I don't know 



41 * Has this pupil ever received instruction in 
basic academic skills as part of the regular 
school program? 

O Yes ^ , 

O No (Go to Q. 44) 



42* Is this pupil currently reGe4v4fig instruction 
in basic-academic skills as part of the regular 
school prc^ram? 

-P Yes 

6 No {Gx> to Q. 44) 



43* Approximately how much i\me in this pupil's 
week is speiit in instruction in basic ocademic 
skills? 

Hours 



44* Approximately what percent of this pupil's day is spent in 
regular classes? (Mark one) 

O None 

O- 1-25% 

O 26-50% 

O 51-75% 

O Over 75% but not oil 

O 100% ((?o toQ. 46) _ 

45* Approximately what percent of this pupil's day^ls spent In 
a special class for the handicapped? (Mark one) 

O None — 

O 1-25% 

O 26-5(5% 

O 51-75% 

O Over 75% but not oil 

O 100% 



46. What services designed to meet the needs of 
handicapped children are provided to this 
' pupil? (Mark all that apply) ^ 

OOivgnQstic assessment 

O Guidance and counseling specifically for the 

hafndlcappod 
O^i^tlonai and social development skills 
OTutorIng 

O Language development ikilii 
O Speech therapy 

O Auditory training , 

O Visual motor training o.^ \ 

O Physical therapy 

O Training In mobility skills 

O Training in se{f*care skills 

O General health care 

OSptcial tronsportation ^ ^ j 

4 O Special resources (e.g., itinerant tutor^f afdes. etc.) 
O Other special services (e.g., pro^rTslon for flexible 
scheduling) 

O Other (Specify): ^ 



47. 



What special equipment or materials are used 
in the services for the handicapped provided 
to this pupil? (Mark all that apply) 



O 
O 

o 
o 

o 
o 
o 
o 
o 



None 

Programmed learning materials 
Audiovisual materialsjsr equipment 
Special equipment and materials for 

speech therapy 
Sensory aids 

Physical therapy equipment ond supplies 
Mobility aids 

Special play equipment and materials 
Other (Specify): 



COMMENTS 



You have completed this questionnaire* However, 
if you have any comments you wouki like to make 
regarding the clarity of the questions, definitions, 
and instructions used in this questionnaire, please 
use the rest of this page, THANK YOU FOR 
YOUR HEtP. 
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